Korištenje
| Verzija | Modul |
|---|
| 4.1.5 | siesta/4.1.5 |
% module load siesta/4.1.5
% siesta
Siesta Version : 4.1.5
Architecture : unknown
Compiler version: ifort (IFORT) 19.0.0.117 20180804
Compiler flags : mpifort -O2 -fPIC -fp-model source -qopenmp -I/apps/netcdf-fortran/4.5.3/include/
PP flags : -DFC_HAVE_ABORT -DMPI -qopenmp -DNCDF -DNCDF_4 -DNCDF_PARALLEL -DSIESTA__METIIS -DSIESTA__ELPA -I/apps/elpa/2020.11.001/avx2/include/elpa-2020.11.001/modules -DSIESTA__DIAG_2STAGE
Libraries : libsiestaLAPACK.a libsiestaBLAS.a libfdict.a libncdf.a -qopenmp -mkl=paralleel -i8 -lmkl_core -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64 -lmkl_sequential -lmkl_blas95_lp64 -L/apps/netcdf-c/4.7.4/lib/ -L/apps/netcdf-fortran/4.5.3/lib/ -L/apps/hdf5/1.12.0/lib/ -lnetcdff -lnetcdf -lhdf5_fortran -lhdf5 -lz -L/apps/metis/5.1.0/lib/ -lmetis -L/apps/elpa/2020.11.001/avx2/lib/ -lelpa
PARALLEL version
NetCDF-4 support
NetCDF-4 MPI-IO support
METIS ordering support
... |
Primjer skripte za opis posla:
#$ -N siesta-parallel
#$ -q p28.q
#$ -pe *mpi 16
#$ -cwd
module load siesta/4.1.5
export OMP_NUM_THREADS=4
mpirun --bind-to socket -hostfile $TMPDIR/machines -np 4 siesta < fe.fdf > fe.out |
Aplikacija se širi hibridno procesima (MPI paradigma) i dretvama (OpenMP paradigma). Varijabla okruženja OMP_NUM_THREADS definira koliko dretvi će svaki MPI proces kreirati. Iz tog razloga umnožak procesa i dretvi mora odgovarati zatraženom broju slotova definiranim u paralelnom okruženju. U primjeru je to 16 = 4 (MPI procesa) x 4 (OpenMP dretve) |
Instalacija
Programski paket SIESTA je pripremljen s kompajlerom Intel 2019, Intel MKL Scalapack i MPI implementacijom OpenMPI 3.0.0. Instalacija je obavljena pripremom pomoćnih biblioteka na koje se paket oslanja, dopremanjem izvornog koda, učitavanjem potrebnih modula za kompajler i MPI paralelizaciju te kompajliranjem samog paketa.
Priprema i instalacija se vršila prema uputstvima (uz male preinake specifične za Isabellu): https://gitlab.com/siesta-project/siesta/-/releases/v4.1.5/downloads/siesta.pdf
Izvorni kod se može pronaći na sjedištu: https://gitlab.com/siesta-project/siesta
Priprema
HD5
https://www.hdfgroup.org/downloads/hdf5/source-code/
module load mpi/openmpi3-intel-x86_64
CC=mpicc CXX=mpicxx FC=mpifort F90=mpif90 ./configure --prefix=/apps/hdf5/1.12.0 --enable-parallel --enable-fortran
make -j8
make install |
NetCDF
https://parallel-netcdf.github.io/
module load mpi/openmpi3-intel-x86_64
./configure --prefix=/apps/pnetcdf/1.12.2 --enable-shared
make -j8
make install |
https://www.unidata.ucar.edu/downloads/netcdf/
module load mpi/openmpi3-intel-x86_64
LD_LIBRARY_PATH="/apps/hdf5/1.12.0/lib/:/apps/pnetcdf/1.12.2/lib:$LD_LIBRARY_PATH"
LIBS="-lhdf5_hl -lhdf5" LDFLAGS="-L/apps/hdf5/1.12.0/lib
-L/apps/pnetcdf/1.12.2/lib" CC=mpicc CPPFLAGS="-I/apps/hdf5/1.12.0/include
-I/apps/pnetcdf/1.12.2/include" CXX=mpicxx ./configure
--prefix=/apps/netcdf-c/4.7.4/ --enable-pnetcdf
make -j8
make install |
module load mpi/openmpi3-intel-x86_64
LD_LIBRARY_PATH="/apps/netcdf-c/4.7.4/lib/:$LD_LIBRARY_PATH" CC=mpicc
CXX=mpicxx FC=mpifort F90=ifort CPPFLAGS="-I/apps/netcdf-c/4.7.4/include/"
LDFLAGS="-L/apps/netcdf-c/4.7.4/lib/" ./configure
--prefix=/apps/netcdf-fortran/4.5.3/
make -j8
make install |
METIS
http://glaros.dtc.umn.edu/gkhome/metis/metis/download
make config prefix=/apps/metis/5.1.0/ cc=icc
make install |
ELPA
https://elpa.mpcdf.mpg.de/elpa-tar-archive
module load mpi/openmpi3-intel-x86_64
LIBS="-qopenmp -mkl=parallel -i8 -lmkl_core -lmkl_scalapack_lp64
-lmkl_blacs_openmpi_lp64 -lmkl_sequential -lmkl_blas95_lp64"
./configure --prefix=/apps/elpa/2020.11.001/avx2
make -j8
make install |
MUMPS
http://mumps.enseeiht.fr/index.php?page=dwnld#form
module load mpi/openmpi3-intel-x86_64
cd MUMPS_5.3.5
cp Make.inc/Makefile.INTEL.PAR Makefile.inc
make all
cp -R lib/ include/ /apps/mumps/5.3.5/ |
Kompajliranje i instalacija SIESTA
wget -c https://gitlab.com/siesta-project/siesta/-/package_files/6659062/download -O siesta-4.1.5.tar.gz
tar -v -xzf siesta-4.1.5.tar.gz
|
cd siesta-4.1.5/Obj/
sh ../Src/obj_setup.sh
cp intel.make arch.make
module load mpi/openmpi3-intel-x86_64 |
Datoteku arch.make je potrebno editirati i dodati postavke za MPI paralelizaciju, Intel kompajler te povezivanje na pomoćne biblioteke:
# parallel compile @ Isabella
CC = mpicc
FPP = $(FC) -E -P
FC = mpifort
FC_SERIAL = ifort
FFLAGS = -O2 -fPIC -fp-model source -qopenmp
MPI_INTERFACE=libmpi_f90.a
MPI_INCLUDE=.
FPPFLAGS += -DMPI -qopenmp
LIBS += -qopenmp -mkl=parallel -i8 -lmkl_core -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64 -lmkl_sequential -lmkl_blas95_lp64
LIBS += -L/apps/netcdf-c/4.7.4/lib/ -L/apps/netcdf-fortran/4.5.3/lib/ -L/apps/hdf5/1.12.0/lib/ -lnetcdff -lnetcdf -lhdf5_fortran -lhdf5 -lz
# fdict
COMP_LIBS += libfdict.a
# ncdf
COMP_LIBS += libncdf.a
FPPFLAGS += -DNCDF -DNCDF_4 -DNCDF_PARALLEL
FFLAGS += -I/apps/netcdf-fortran/4.5.3/include/
# metis
LIBS += -L/apps/metis/5.1.0/lib/ -lmetis
FPPFLAGS += -DSIESTA__METIS
# elpa
LIBS += -L/apps/elpa/2020.11.001/avx2/lib/ -lelpa
FPPFLAGS += -DSIESTA__ELPA -I/apps/elpa/2020.11.001/avx2/include/elpa-2020.11.001/modules
# mumps
LIBS += -L/apps/mumps/5.3.5/lib/ -lzmumps -lmumps_common -lpord
FPPFLAGS += -DSIESTA__MUMPS
FFLAGS += -I/apps/mumps/5.3.5/include/ |
make -j8
cp siesta /apps/siesta/4.1.5/avx2 |
cd siesta-4.1.5/Util
find . -name Makefile -exec sed -i 's/$(LDFLAGS)/$(LDFLAGS) -qopenmp/g' {} \;
./build_all
find . -type f -executable -exec cp {} /apps/siesta/4.1.5/avx2 \;
ls /apps/siesta/4.1.5/avx2/
h2o/ eig2bxsf* fractional* ionplot.sh* pblas_prb* simple* surf.py*
2dplot.py* eigfat2plot* get_chem_labels* macroave* pi3* simple_mpi_parallel* tbtrans*
Eig2DOS* f2fmaster* gnubands* md2axsf* plstm* simple_mpi_serial* ts2ts*
blacs_prb* f2fslave* grid1d* mixps* plsts* simple_pipes_parallel* tselecs.sh*
ccViz* fat* grid2d* mprop* protoNEB* simple_pipes_serial* vib2xsf*
clean_all.sh* fcbuild* hs2hsx* optical* rho2xsf* simple_sockets_parallel* vibra*
denchar* fdf2grimme* hsx2hs* optical_input* sies2arc* simple_sockets_serial* xv2xsf*
driver* fmpdos* int_explorer* para* siesta* spin_texture*
|