Here we illustrate how to use GSVA in an HPC environment.
GSVA 2.7.8
License: Artistic-2.0
Splitting GSVA calculations in parallel on a multicore shared-memory computer is
directly supported in the functions gsva(), gsvaRowNorm(), gsvaColRanks()
and gsvaColScores(), by using the BPPARAM parameter with backends such
as BiocParallel::MulticoreParam() or BiocParallel::SnowParam(). However,
running GSVA in a high-performance computing (HPC) environment, distributing the
computational load across multiple independent nodes that do not share their
main memory, requires using some additional functions for that purpose. In this
vignette we illustrate this use case assuming our HPC environment uses the
SLURM workload manager. Other workload managers
can be used, please consult the documentation of the BiocParallel
package for more information, or open an
issue in the GSVA GitHub
repo if you need help with a different workload manager.
In essence, we will be using the three-step GSVA pipeline, illustrated in the
scRNA-seq vignette, with the additional parameters first and last, which
allow the algorithm to run on a subset of rows (gsvaRowNorm()) or columns
(gsvaColRanks() and gsvaColScores()) of the input data, storing the
intermediate results using on-disk data structures, and merging them into a
final single output object that will store the enrichment scores using also
an on-disk data structure. We will not have to worry about how to split the
input data and merge the intermediate results. These two tasks will be executed
by the functions gsvaMap() and gsvaReduce().
Let’s assume we have run the first part of the scRNA-seq vignette, and we have
obtained a SingleCellExperiment object called sce with filtered and
normalized expression values, and a collection of gene sets stored in an object
called gsets.
We first build a parameter object using the function gsvaParam(), as we would
normally do.
gsvapar <- gsvaParam(sce, gsets)We would then build a BatchtoolsParam object, as described in the
corresponding
vignette
of the BiocParallel package. The GSVA package, however, provides
a convenience function called gsvaBatchtoolsSlurmParam() that will build a
BatchtoolsParam object with some sensible settings for running GSVA in an HPC
environment with a SLURM workload manager.
gsvabtpar <- gsvaBatchtoolsSlurmParam(partition="short", nodes=10,
ncpus_per_task=4)In this example, we are distributing the computational load across 10 nodes, where
in each of them calculations will be done in parallel using 4 CPU cores, sending
the corresponding jobs to a partition called short. By default, the function
gsvaBatchtoolsSlurmParam() has a first argument dir="GSVAOUTPUT", which will
create a directory called GSVAOUTPUT in the current working directory, where
the intermediate results will be stored. The user can change this directory by
specifying a different value for the dir argument, but it is important that this
directory is accessible from all nodes in the HPC environment. The user is also
responsible for deleting the contents of this directory after the GSVA calculations
are finished.
The next step is to run the function gsvaMap(), which will submit the jobs to the
SLURM workload manager, and return a list object, where each of its elements will be
of the same class as the input expression data, but with the intermediate results
stored in an on-disk data structure.
gsvamaprnorm <- gsvaMap(gsvaRowNorm, gsvapar, BTPARAM=gsvabtpar)The final step is to merge the intermediate results into a single object, using
the function gsvaReduce(), which in this example will return a
SingleCellExperiment object with the row-normalized values stored in an
assay called gsvarnorm.
gsvarnorm <- gsvaReduce(gsvamaprnorm)To complete the three-step GSVA pipeline, we would then calculate the column
rank values using the function gsvaColRanks(), and finally the GSVA scores
using the function gsvaColScores(), as we would normally do in a single-node
environment, but using gsvaMap() and gsvaReduce(), this time for a more
compact coding in single line of code each.
gsvaranks <- gsvaReduce(gsvaMap(gsvaColRanks, gsvarnorm, BTPARAM=gsvabtpar))
gsvaes <- gsvaReduce(gsvaMap(gsvaColScores, gsvaranks, BTPARAM=gsvabtpar))Since the calculations of the column ranks and enrichment scores are both done
on the columns of the input data, we can reuse the output of gsvaMap()
obtained from the rank calculations, directly to the input of gsvaMap() for
the enrichment score calculations, without having to reduce the intermediate
results into a single object, avoiding the overhead of that unnecessary
reduction.
gsvamapranks <- gsvaMap(gsvaColRanks, gsvarnorm, BTPARAM=gsvabtpar)
gsvaes <- gsvaReduce(gsvaMap(gsvaColScores, gsvamapranks, BTPARAM=gsvabtpar))The current implementation using the BatchtoolsParam backend makes the
gsvaMap() function blocking, waiting for all jobs to finish before returning
a list object with the intermediate results. There are currently two ways to
circumvent this limitation. The first one is to use a terminal multiplexer such
as tmux, log interactively into a node of
the HPC environment using a job submission command such as:
srun --mpi=none --mem=10G --nodes=1 --ntasks-per-node=1 --partition=short --pty bash -istart an R session, call interactively the gsvaMap() function and detach the
tmux session to be able to log out and leave the jobs running in the background.
You may reattach to the tmux session later to check the progress of the jobs or
fetch the results. Note that the partition where the interactive job is
submitted must have a walltime limit that is long enough to allow the jobs to finish.
The second way is to submit a job with the R script that calls the gsvaMap()
function, which will submit the jobs to the SLURM workload manager.
The current implementation of gsvaMap() and gsvaReduce() does not yet handle
partial completion of the jobs. So, if any of the jobs fail, the user will have
to resubmit the entire calculation. We are working to enable the resubmission of
only the failed jobs.
Here is the output of sessionInfo() on the system on which this document was
compiled running pandoc 2.7.3:
sessionInfo()
R version 4.6.1 (2026-06-24)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 24.04.4 LTS
Matrix products: default
BLAS: /home/biocbuild/bbs-3.24-bioc/R/lib/libRblas.so
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.12.0 LAPACK version 3.12.0
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_GB LC_COLLATE=C
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
time zone: America/New_York
tzcode source: system (glibc)
attached base packages:
[1] stats4 stats graphics grDevices utils datasets methods
[8] base
other attached packages:
[1] sva_3.61.0 BiocParallel_1.47.0
[3] genefilter_1.95.0 mgcv_1.9-4
[5] nlme_3.1-169 RColorBrewer_1.1-3
[7] edgeR_4.11.4 limma_3.69.2
[9] GSVAdata_1.49.0 SummarizedExperiment_1.43.0
[11] GenomicRanges_1.65.0 Seqinfo_1.3.0
[13] MatrixGenerics_1.25.0 matrixStats_1.5.0
[15] GSEABase_1.75.0 graph_1.91.0
[17] annotate_1.91.0 XML_3.99-0.23
[19] org.Hs.eg.db_3.23.1 AnnotationDbi_1.75.0
[21] IRanges_2.47.2 S4Vectors_0.51.5
[23] Biobase_2.73.1 BiocGenerics_0.59.8
[25] generics_0.1.4 GSVA_2.7.8
[27] BiocStyle_2.41.0
loaded via a namespace (and not attached):
[1] blob_1.3.0 Biostrings_2.81.3
[3] fastmap_1.2.0 SingleCellExperiment_1.35.1
[5] digest_0.6.39 rsvd_1.0.5
[7] lifecycle_1.0.5 survival_3.8-6
[9] statmod_1.5.2 KEGGREST_1.53.4
[11] RSQLite_3.53.3 magrittr_2.0.5
[13] compiler_4.6.1 rlang_1.2.0
[15] sass_0.4.10 tools_4.6.1
[17] yaml_2.3.12 knitr_1.51
[19] S4Arrays_1.13.0 bit_4.6.0
[21] DelayedArray_0.39.3 abind_1.4-8
[23] HDF5Array_1.41.0 memuse_4.2-3
[25] grid_4.6.1 beachmat_2.29.0
[27] xtable_1.8-8 Rhdf5lib_2.1.0
[29] tinytex_0.60 cli_3.6.6
[31] rmarkdown_2.31 crayon_1.5.3
[33] otel_0.2.0 httr_1.4.8
[35] rjson_0.2.23 DelayedMatrixStats_1.35.0
[37] DBI_1.3.0 cachem_1.1.0
[39] rhdf5_2.57.1 splines_4.6.1
[41] parallel_4.6.1 BiocManager_1.30.27
[43] XVector_0.53.0 vctrs_0.7.3
[45] Matrix_1.7-5 jsonlite_2.0.0
[47] bookdown_0.47 BiocSingular_1.29.0
[49] bit64_4.8.2 irlba_2.3.7
[51] magick_2.9.1 locfit_1.5-9.12
[53] h5mread_1.5.0 jquerylib_0.1.4
[55] glue_1.8.1 codetools_0.2-20
[57] ScaledMatrix_1.21.0 pillar_1.11.1
[59] htmltools_0.5.9 rhdf5filters_1.25.0
[61] R6_2.6.1 sparseMatrixStats_1.25.0
[63] evaluate_1.0.5 lattice_0.22-9
[65] png_0.1-9 SpatialExperiment_1.23.0
[67] memoise_2.0.1 bslib_0.11.0
[69] Rcpp_1.1.1-1.1 SparseArray_1.13.2
[71] xfun_0.59 pkgconfig_2.0.3