Advanced Research Computing
Spack Package Installations
Spack is a unique HPC focused package manager that allows users to install different variations of the same package which can include different versions of the package, compiler builds, dependency options and much more. Spack specs are used to download a specific package with a specific build configuration.
More information of Spack can be accessed here at the: Spack Documentation Page as well as the GitHub documentation page.
Installing Spack Packages
Spack may be loaded like a regular module using the module load spack command followed by the spack install/uninstall commands to either install or uninstall specific packages with their desired specs.
Though packages may be installed directly with the commands shown above, some software may take up a lot of resources. It is recommended that users utilize the spack_install script provided to run their installs on compute nodes to evade the potential issue of excessive resource usage on the login nodes. The spack_install script submits a job for the user which will install the specific packages passed into the parameters.
Note: Though the script loads the Spack module for the install, users must then manually load Spack themselves to access the software.
Users may specify a package by itself or provide additional specs they would like to add to the installation process. This process will then submit a job on behalf of the user and once spack in loaded, the software will be ready to be used.
[abc123@wind ~]$ ./spack_install gzip INFO: this module cannot be purged or unloaded, start a new shell session to unload Input spec -------------------------------- - gzip Concretized -------------------------------- [+] gzip@1.12%gcc@8.5.0 build_system=autotools arch=linux-rhel8-x86_64_v2 Install gzip? (yes/no): yes
If a specific version of a package is preferred where multiple arguments are needed, enclosing the specifications into quotes allows for the install script to process everything that needs to be installed.
[abc123@wind ~]$ ./spack_install "py-mpi4py ^python@3.9" Input spec -------------------------------- - py-mpi4py - ^python@3.9 Concretized -------------------------------- - py-mpi4py@3.1.4%gcc@8.5.0 build_system=python_pip arch=linux-rhel8-x86_64_v 2
The user may also specify a node they want their installation to run on using the -n flag with the node name following.
[abc123@wind ~]$ ./spack_install gzip -n cn32
Additional Spack Commands
After the desired package is installed users may find these commands helpful with managing specific packages
spack find <PACKAGE NAME>: allows users to find a specific package they may have installed. If no package name is specified then spack finds all packages the user may have installed
[abc123@wind ~]$ spack install gzip ==> Installing gzip-1.12-aaksfrav72c234tchtdw5pf57l4ckl2a ==> gzip: Successfully installed gzip-1.12-aaksfrav72c234tchtdw5pf57l4ckl2a
spack list: shows all available packages that Spack offers for installation
spack spec <PACKAGE NAME>: shows the specific build configurations of the package
[abc123@wind ~]$ spack spec gzip Input spec -------------------------------- gzip] Concretized -------------------------------- gzip@1.12%gcc@8.5.0 build_system=autotools arch=linux-rhel8-x86_64_v2
spack info <PACKAGE NAME>: gets more information on a package.
[abc123@wind ~]$ spack info gzip AutotoolsPackage: gzip Description: GNU Gzip is a popular data compression program originally written by Jean-loup Gailly for the GNU project.
spack find <PACKAGE NAME>: command goes through all the installed packages in the database. Spack find will only go through packages installed in the environment the user is in.
[abc123@wind ~]$ spack find gzip -- linux-rhel8-x86_64_v2 / gcc@8.5.0 ---------------------------- gzip@1.12 ==> 1 installed package
Spack Environments
Spack environments allows for users to work with different packages separately. Unlike Conda environments, Spack environments are not isolated. More information on Spack environments can be found here.
spack env create -d ./<ENV NAME>: creates an environment in the specified directory
[abc123@wind ~]$ spack env create -d ./mydir ==> Created environment in /home/abc123/mydir ==> You can activate this environment with: ==> spack env activate /home/abc123/mydir
spack activate <ENV NAME>: activates a created spack environment. A spack environment can be deactivated the same way as well.
[abc123@wind ~]$ spack env activate /home/abc123/mydir
An abstract spec is only a partially specified configuration of a package. This means that the spec can potentially be used to describe multiple builds of the same package. Abstract specs are useful because the user is able to add these spack to an environment without installing them. The spack add command is environmentally sensitive, meaning the spec will only be added to the current environment.
[abc123@wind ~/mydir ]$ spack add gzip ==> Adding gzip to environment /home/abc123/mydir
Even though the specs have been added to an environment with spack add, they are still unable to be processed and installed. The spec has to complete the previous partially specified build of the package (when spack add was used). This can be done with the a process called concretization.
[abc123@wind ~/mydir ]$ spack concretize ==> Concretized gzip [+] aaksfra gzip@1.12%gcc@8.5.0 build_system=autotools arch=linux-rhel8-x86_64_v2 ==> Updating view at /home/abc123/mydir/.spack-env/view
Once the package has been added and concretized, it is ready to be installed. Using the spack find command, it can be seen that the specific package has gained all its required dependencies but it still not actively installed into the environment.
$ spack find -c ==> In environment /home/abc123/mydir ==> Root specs ip ==> Concretized roots -- linux-rhel8-x86_64_v2 / gcc@8.5.0 ---------------------------- ip@3.3.3 ==> 0 installed packages
Running the spack install command allows spack to install the packages added and concretized in your environment. This will install the desired package plus any dependencies it might of required. Running the spack find command again after the installation will give you everything you have now installed into your environment.
$ spack install ==> Installing environment /home/abc123/mydir