---
title: "LegATo Tutorial"
author: 
- name: Aubrey R. Odom
  affiliation: 
  - Program in Bioinformatics, Boston University, Boston, MA
  email: aodom@bu.edu
date: '`r format(Sys.Date(), "%B %e, %Y")`'
package: LegATo
output: html_document
vignette: >
  %\VignetteIndexEntry{LegATo Tutorial}
  %\VignetteEncoding{UTF-8}
  %\VignetteEngine{knitr::rmarkdown}
editor_options: 
  chunk_output_type: console
---

# Introduction to LegATo

## Streamlining longitudinal microbiome profiling in Bioconductor

Microbiome time-series data presents several distinct challenges, including complex covariate dependencies and a variety of longitudinal study designs. Furthermore, while several individual tools have been created to aid in longitudinal microbiome data analysis, an all-inclusive R-based toolkit has remained elusive. 

To address these concerns, we have created LegATo (Longitudinal mEtaGenomic Analysis TOolkit). LegATo is a suite of open-source software tools for longitudinal microbiome analysis that is extendable to several different study forms, all while promising optimal ease-of-use for researchers.  This toolkit will allow researchers to determine which microbial taxa are affected over time by perturbations such as onset of disease or lifestyle choices, and to predict the effects of these perturbations over time, including changes in composition or stability of commensal bacteria.

Currently, LegATo entertains a number of data cleaning, aggregation, modeling and testing procedures. As we develop or learn of new methods, we will add to the toolkit accordingly. We will soon add hierarchical clustering tools and multivariate generalized estimating equations (JGEEs) to adjust for the compositional nature of microbiome data.

# Getting started with LegATo

## Compatibility with `MultiAssayExperiment` and `SummarizedExperiment` objects

The convenience of LegATo is in part credited to the integration of `MultiAssayExperiment` and `SummarizedExperiment` objects. These are reliable and clean data structures developed by the \emph{Bioconductor team} as part of the `r BiocStyle::Biocpkg("MultiAssayExperiment")` and `r BiocStyle::Biocpkg("SummarizedExperiment")` packages.

The `MultiAssayExperiment` container concisely stores a `SummarizedExperiment` object, which aggregates data matrices along with annotation information, metadata, and reduced dimensionality data (PCA, t-SNE, etc.). To learn more about proper usage and context of these objects, you may want to take a look at the [MultiAssayExperiment package vignette](https://bioconductor.org/packages/release/bioc/vignettes/MultiAssayExperiment/inst/doc/MultiAssayExperiment.html) and [SummarizedExperiment package vignette](https://bioconductor.org/packages/release/bioc/vignettes/SummarizedExperiment/inst/doc/SummarizedExperiment.html).

To install these two packages, run the following code:
```{r MAE_SE_install, eval = FALSE}
if (!requireNamespace("BiocManager", quietly = TRUE))
    install.packages("BiocManager")

BiocManager::install("MultiAssayExperiment")
BiocManager::install("SummarizedExperiment")
```

## Installation and setup

To install the development version of LegATo from Github, run the following code:

```{r load_LegATo, eval = FALSE}
BiocManager::install("wejlab/LegATo")
```

Now we'll load the LegATo package into our library:
```{R}
library(LegATo)
```

For the purposes of this vignette, we'll also load the `ggeffects` and `emmeans` library:
```{R}
library(ggeffects)
library(emmeans)
```

# Formatting input data for use with LegATo

LegATo functions expect data properly formatted in a `MultiAssayExperiment` container, with naming conventions that are compatible with `r BiocStyle::Biocpkg("animalcules")`. Obtaining properly formatted data can be achieved easily via the `create_formatted_MAE()` function from the LegATo package, as detailed below.

## A note on compatibility with other packages

If you've analyzed your metagenomic or 16S data via the [MetaScope package](https://wejlab.github.io/metascope-docs/), you can aggregate your sample outputs with user-input metadata using the `MetaScope::convert_animalcules()` or `MetaScope::convert_animalcules_silva()` functions prior to loading your data into LegATo. These functions create a taxonomy table and collate all data into a LegATo- and animalcules-compatible `MultiAssayExperiment` object.

You can also format your data analyzed elsewhere (e.g., PathoScope 2.0 or Kraken) with the [animalcules package](https://wejlab.github.io/animalcules-docs/) in the upload step with the R Shiny app, and then select "Download Animalcules File" to obtain a `MAE` object that can be used with LegATo.

## Requirements for using `create_formatted_MAE()`

For the purposes of this example, we'll look at some example inputs unrelated to the main HIV-E infant data:

* The `counts` table is composed of raw microbial counts. There is one row for each taxon, and one column for each sample.
* The `tax` table is composed of taxonomy lineages. There is one row for each taxon, and any number of columns each taxonomic level (e.g., superkingdom, genus, species). The lowest level of taxonomy serves as the rownames for the table, and is also stored in its own column. Taxonomic levels increase in granularity from left to right.
* The `sample` table contains entries for each sample, with unique sample IDs on the rows. The columns are metadata for the samples, such as subjects or units on which repeated measures were taken, groupings, pairings, and covariates.
* `rownames(counts) == rownames(tax)`
* `colnames(counts) == colnames(sample)`

```{R}
counts <- system.file("extdata", "counts.csv", package = "LegATo") |>
  read.csv(row.names = 1) |>
  dplyr::rename_with(function(x) stringr::str_replace(x, "\\.", "-"))
tax <- system.file("extdata", "tax.csv", package = "LegATo") |> read.csv(row.names = 1)
sample <- system.file("extdata", "sample.csv", package = "LegATo") |> read.csv(row.names = 1)
```

Now we'll look at the formatting of each:
```{R}
ndim <- 5
counts[seq_len(ndim), seq_len(ndim)] |>
  knitr::kable(caption = "Counts Table Preview",
               label = NA)

tax[seq_len(ndim), ] |>
  knitr::kable(caption = "Taxonomy Table Preview",
               label = NA)

sample[seq_len(ndim), ] |>
  knitr::kable(caption = "Sample Table Preview",
               label = NA)
```

## Formatting your inputs with `create_formatted_MAE()`

Once your data are formatted correctly, you can easily use `create_formatted_MAE()` like so:
```{R}
output <- create_formatted_MAE(counts_dat = counts,
                               tax_dat = tax,
                               metadata_dat = sample)

class(output)
MultiAssayExperiment::assays(output)
SummarizedExperiment::assays(output[["MicrobeGenetics"]])
```

If your data is in the format of a TreeSummarizedExperiment, you can call `create_formatted_MAE(tree_SE)` to create a `MAE` output that is compatible with the package.

## Adding information to your metadata later in your analysis

If information needs to be added to your data object at some later point in the analysis, it is easiest to manipulate the raw data objects (potentially via `parse_MAE_SE()`) and then recreate the `MAE` object with `create_formatted_MAE()`.

# Example data

To illustrate the capabilities of LegATo, we will turn to a published dataset from the following paper:

Odom AR, Gill CJ, Pieciak R et al. Characterization of longitudinal nasopharyngeal microbiome patterns in maternally HIV-exposed Zambian infants [version 2; peer review: 2 approved with reservations]. Gates Open Res 2024, 6:143 (https://doi.org/10.12688/gatesopenres.14041.2)

The raw dataset is archived in Zenodo:

*Zenodo: Underlying data for ‘Characterization of longitudinal nasopharyngeal microbiome patterns in maternally HIV-exposed Zambian infants’. https://doi.org/10.5281/zenodo.725531324*

Further details on how the dataset was altered for inclusion in this package are provided [here](https://github.com/wejlab/LegATo/blob/main/inst/script/extdata_explanations.Rmd).

### Example data context and structure

The example data consists of 167 NP swabs of healthy HIV-exposed, uninfected (HEU; n=10) infants and their HIV(+) mothers and HIV-unexposed, uninfected (HUU; n=10) infants and their HIV(-) mothers. A total of 7 samples were collected per infant, with some missingness in the data. These swabs were identified from a sample library collected in Lusaka, Zambia between 2015 and 2016. 

The analysis objective is to parse the association between the NP resident bacteria and infant HIV exposure during the first 3.5 months (14 weeks) of life, a critical time in microbiome maturation.

All data was processed with `PathoScope 2.0`, and the sample outputs were aggregated with the `animalcules` R package.

For this analysis, we will work with the infant data only and subset the data accordingly.

```{R}
dat <- system.file("extdata", "MAE.RDS", package = "LegATo") |>
  readRDS()

dat_subsetted <- MultiAssayExperiment::subsetByColData(dat,
                                                       dat$MothChild == "Infant")
```

This leaves us with `r dim(parse_MAE_SE(dat_subsetted)$sam)[1]` samples in our analysis.

# Initial data exploration

## `MultiAssayExperiment (MAE)` manipulation

`LegATo` has several functions for manipulating `MAE` objects, delineated below:

### Clean up animalcules-formatted `MAE`s

If your data were aggregated via the `animalcules` package, as was done for the example data, the samples were aggregated at the strain taxonomic level and have residual taxonomy IDs present. The `clean_MAE()` function cleans up data suffering from these issues.

```{R}
dat_cleaned <- clean_MAE(dat_subsetted)
```

### Filter MAE

Many metagenomic pipelines identify taxon abundances at extremely small levels, which can be noisy to deal with in an analysis. The `filter_MAE` function smoothly transforms reads belonging to taxa with an overall genera threshold under the `filter_prop` (filter proportion) argument, which we will set as 0.005.

```{R}
dat_filt_1 <- filter_MAE(dat_cleaned, relabu_threshold = 0.05, occur_pct_cutoff = 10)
```

The `filter_animalcules_MAE` function utilizes the same filtering mechanism as in the animalcules package:

```{R}
dat_filt <- filter_animalcules_MAE(dat_cleaned, 0.05)
```

### Parse MAE to extract data

If you want to take a closer look at your data, you can easily extract it into the counts, taxonomy, and sample metadata tables using the `parse_MAE_SE()` function.

```{R}
parsed <- parse_MAE_SE(dat_filt, type = "MAE")

parsed$counts[seq_len(5), seq_len(5)] |> 
  knitr::kable(caption = "Counts Table")

parsed$sam[seq_len(5), ]  |>
  knitr::kable(caption = "Sample Metadata")

parsed$tax[seq_len(5), ] |> 
  knitr::kable(caption = "Taxonomy Table")

```

### Summarizing microbial counts data

It is fairly convenient to summarize the average number of reads for a report with `get_summary_table()`. The table groups by user-provided discrete covariates.
```{R}
group_vars <- c("HIVStatus", "MothChild")
get_summary_table(dat_filt, group_vars) |>
  knitr::kable(caption = "Summary Table", label = NA)
```

The `get_top_taxa` function outputs a `data.frame` that lists taxa in order of relative abundance.
```{R}
best_genus <- get_top_taxa(dat_filt, "genus")
best_genus |> knitr::kable(caption = "Table of genera, ranked by abundance")
```

### Other data manipulation functions

If you want to conduct your own analyses, the `get_long_data()` function will prove convenient. The `get_stacked_data()` function can be used for certain visualizations that utilize a relative abundance aggregation approach.

```{R}
longdat <- get_long_data(dat_filt, "genus", log = TRUE, counts_to_CPM = TRUE)

stackeddat <- get_stacked_data(dat_filt, "genus", covariate_1 = "HIVStatus",
                               covariate_time = "timepoint")
```

## Visualizing Data

There are several plots by which we can visualize changes in relative abundance over time, accounting for a given covariate. In this case, we are interested in HIV exposure.

We'll select a palette using `paletteer`.
```{R}
this_palette <- c("#709AE1", "#8A9197", "#D2AF81", "#FD7446", "#D5E4A2", "#197EC0", "#F05C3B", "#46732E",
                  "#71D0F5", "#370335", "#075149", "#C80813", "#91331F", "#1A9993", "#FD8CC1") |>
  magrittr::extract(seq_len(nrow(best_genus)))
```

### Alluvial plots

Alluvial diagrams illustrate individual taxa as stream fields that change position at different time points. The height of a stream field represents the relative abundance of that taxon. At a given time point, stream fields are ranked from the highest to lowest abundance (top to bottom). These can be constructed with `plot_alluvial()`.
```{R}
plot_alluvial(dat = dat_filt, 
              taxon_level = "genus", 
              covariate_1 = "HIVStatus",
              covariate_time = "timepoint",
              palette_input = this_palette,
              subtitle = "Alluvial plot")
```

In context, this plot indicates a high abundance of Staphylococcus in both the control and HIV-E infants in the first two weeks of life, but other genera quickly overtake **Staphylococcus** in dominance as shown by the changing rank of the streams for each time point. Overall, **Streptococcus** seems to have a far greater presence in the HIV-E infants' nasopharyngeal microbiota

### Spaghetti plots

We can create spaghetti or volatility plots to elucidate changes over time on a sample level for a given taxon. This is advantageous as other visualization methods are often aggregates of multiple samples and lack granularity. These plots can be created with `plot_spaghetti()`.

```{R, results = "asis", message = FALSE}
  plot_spaghetti(dat = dat_filt,
               covariate_time = "timepoint",
               covariate_1 = "HIVStatus",
               unit_var = "Subject",
               taxon_level = "genus",
               which_taxon = "Staphylococcus",
               palette_input = this_palette,
               title = "Spaghetti Plot",
               subtitle = NULL) +
  ggplot2::xlab("Infant Age (timepoint)") +
  ggplot2::ylab("Relative Abundance (log CPM)")
```

Here we plot changes in the **Staphylococcus** log counts per million (CPM) across time points. The spaghetti plot format clues us into the few outlier infants in the control group that seem to have extremely high counts in the first few weeks of life.

### Stacked bar plots

Stacked bar plots are used here to visualize the relative abundance of microbes at a given taxonomic level in each sample, represented as a single bar, labeled by time point, and plotted within each HIV exposure status group for separate mothers and infant comparisons. Use `plot_stacked_bar`.
```{R}
plot_stacked_bar(dat_filt, "genus", 
             "HIVStatus",
             "timepoint",
             palette_input = this_palette)
```

We see from the stacked bar plots that many of the microbes seem to maintain similar trends in both control and HIV-E infants over time, with **Streptococcus** producing slightly higher abundances in the HIV-E group.

### Stacked area charts

Stacked area charts are similar to stacked bar plots, but provide for continuity between time points. These are created with `plot_stacked_area()`.  

```{R}
plot_stacked_area(dat_filt, "genus", 
                  "HIVStatus",
                  "timepoint",
                  palette_input = this_palette)
```

The stacked area chart is offers a similar method of visualizing the data.

### Heatmaps

With `plot_heatmap`, you can plot a heatmap of a specific microbe to determine changes along one or more covariates. 

```{R}
this_taxon <- parsed$counts |>
  animalcules::upsample_counts(parsed$tax, "genus") |>
  animalcules::counts_to_logcpm()
p <- plot_heatmap(inputData = this_taxon,
                  annotationData = dplyr::select(parsed$sam, "timepoint", "HIVStatus", "pairing"),
                  name = "Data",
                  plot_title = "Example",
                  plottingColNames = NULL,
                  annotationColNames = NULL,
                  colList = list(),
                  scale = FALSE,
                  showColumnNames = FALSE,
                  showRowNames = FALSE,
                  colorSets = c("Set1", "Set2", "Set3", "Pastel1", "Pastel2", "Accent", "Dark2",
                                "Paired"),
                  choose_color = c("blue", "gray95", "red"),
                  split_heatmap = "none",
                  column_order = NULL
)
```

From the heatmap, we find that clustering does not seem to find a clear relationship between genera (on the rows) and samples (on the columns). Three annotations were plotted on the top bar: time point, HIV status, and pairing.

# Longitudinal data analysis

For a concrete analysis of longitudinal microbiome data, `LegATo` provides NMIT, both a paired and unpaired multivariate Hotelling's T-squared test, generalized estimating equations (GEEs) and linear mixed effects models (LMEMs/LMMs).

A brief overview of each method is provided below.

## Nonparametric microbial interdependence test (NMIT)

NMIT is a multivariate distance-based test intended to evaluate the association between key phenotypic variables and microbial interdependence. The test determines longitudinal sample similarity as a function of temporal microbial composition. 
  
The authors thank Yilong Zhang for providing his code for adaptation into LegATo.
  
Citations:

Yilong Zhang, Sung Won Han, Laura M Cox, and Huilin Li. A multivariate distance-based analytic framework for microbial interdependence association test in longitudinal study. Genetic epidemiology, 41(8):769–778, 2017. doi:10.1002/gepi.22065.
```{R}
dat_1 <- filter_MAE(dat_cleaned, 0.05, 10, "species")

NMIT(dat_1, unit_var = "Subject", fixed_cov = "HIVStatus",
     covariate_time = "timepoint",
     method = "kendall", dist_type = "F",
     heatmap = TRUE, classify = FALSE, fill_na = 0)
```

## Hotelling's T^2 Test

Hotelling’s T2 tests can be used to determine whether the microbiome profiles exhibit notable differences or trends across time and groups. We may then use t-tests to identify which genera contributed most to these differences.

For this example, HEU and HUU infants are designated as the two sampling units on which the relative abundances of the p most abundant genera will be measured. 

For paired tests, we chose p = 6 variables to ensure that n < p so that singularity could be avoided and T2 could be properly computed, where n is the number of measurements in a sampling unit. Normality is met by using microbe abundances in log CPM units, which is calculated within the function.

```{R}
test_hotelling_t2(dat = dat_1,
                  test_index = which(dat_filt$MothChild == "Infant" &
                                       dat_filt$timepoint == 6),
                  taxon_level = "genus",
                  num_taxa = 6,
                  paired = TRUE,
                  grouping_var = "HIVStatus",
                  pairing_var = "pairing")
```

Group comparisons can also be tested on unpaired data:
```{R}
test_hotelling_t2(dat = dat_1,
                  test_index = which(dat_filt$timepoint == 0),
                  taxon_level = "genus",
                  num_taxa = 6,
                  grouping_var = "HIVStatus",
                  unit_var = "Subject",
                  paired = FALSE)

```

## Modeling

### Generalized Estimating Equations (GEEs)

Generalized estimating equations (GEEs) as described in Liang and Zeger (1986) and extended by Agresti (2002) have been widely used for modeling longitudinal data, and more recently for longitudinal microbiome data.

For each genus present in the microbial aggregate of samples, we model normalized log CPM relative taxon counts, estimating the effects of time point and HIV exposure status and their interaction, while accounting for the underlying structure of clusters formed by individual subjects. 

```{R}
output <- run_gee_model(dat_1, unit_var = "Subject",
                        fixed_cov = c("HIVStatus", "timepoint"),
                        corstr = "ar1",
                        plot_out = FALSE,
                        plotsave_loc = ".",
                        plot_terms = NULL)

head(output) |> knitr::kable(caption = "GEE Outputs")
```

You can also create plots of the covariates, which will be saved to a folder specified by the user:

```{R, out.width = "50%", fig.align = "center", echo = FALSE}
tempfolder <- tempdir()

# Trying out plotting
output <- run_gee_model(dat_1, unit_var = "Subject",
              taxon_level = "genus",
              fixed_cov = c("HIVStatus", "timepoint"),
              corstr = "ar1",
              plot_out = TRUE,
              plotsave_loc = tempfolder,
              plot_terms = "timepoint",
              width = 6, height = 4, units = "in", scale = 0.7)

list.files(tempfolder) |> head()
```

### Linear Mixed Models

Similarly, you can also run linear mixed-effects models:
```{R}
# If there are issues with matrix being positive definite
# Revisit filtering parameters in filter_MAE
output <- run_lmm_model(dat_1, unit_var = "Subject",
                        taxon_level = "genus",
                        fixed_cov = c("timepoint", "HIVStatus"),
                        plot_out = FALSE,
                        plotsave_loc = ".",
                        plot_terms = NULL)

head(output) |> knitr::kable(caption = "LMM Outputs")
```

# Session Info

```{R}
sessionInfo()
```


