# iCOBRA - Interactive benchmarking of ranking and assignment methods

`iCOBRA` is a package to calculate and visualize performance metrics for 
ranking and binary assignment methods. A typical use case could be, 
for example, comparing methods calling differential expression in 
gene expression experiments, which could be seen as either a ranking 
problem (estimating the correct effect size and ordering the genes by 
significance) or a binary assignment problem (classifying the genes 
into differentially expressed and non-differentially expressed).

`iCOBRA` can be used either directly from the console, or via the 
interactive shiny application (see the function `COBRAapp()`). It can also 
be accessed from the web server [http://imlspenticton.uzh.ch:3838/iCOBRA/](http://imlspenticton.uzh.ch:3838/iCOBRA/)

We have also collected a set of benchmarking data sets, addressing different aspects of genomic data analysis. The collection is reachable via the following link: [http://imlspenticton.uzh.ch/robinson_lab/benchmark_collection/](http://imlspenticton.uzh.ch/robinson_lab/benchmark_collection/)

## Installation

`iCOBRA` can be installed from `Bioconductor` using `BiocManager`:

```
install.packages("BiocManager")
BiocManager::install("iCOBRA")
```

or, optionally, 

```
BiocManager::install("markrobinsonuzh/iCOBRA")
```


## Quick start guide

The `iCOBRA` workflow starts from an object of class `COBRAData`, 
containing (adjusted) p-values and/or scores for a set of features as 
well as the true status of the features. An example data set is provided in 
the package

```
library(iCOBRA)
data(cobradata_example)
```

The function `calculate_performance()` calculates the different performance 
metrics for a `COBRAData` object. By default, all performance metrics are 
calculated, but a subset can be selected using the `aspects` argument.

```
cobraperf <- calculate_performance(cobradata_example, binary_truth = "status",
                                   cont_truth = "logFC", 
                                   aspects = c("fdrtpr", "fdrtprcurve", 
                                               "corr"))
```

Next, the performance metrics are prepared for plotting using the 
`prepare_for_plot()` function. This function defines colors for the 
various methods and can also be used for selecting only a subset of the 
methods for visualization, without having to recalculate the performance metrics.

```
cobraplot <- prepare_data_for_plot(cobraperf, colorscheme = "Set2",
                                   keepmethods = c("voom", "edgeR"))
```

The resulting object can then be used to generate plots of the selected aspects.

```
plot_fdrtprcurve(cobraplot)
plot_corr(cobraplot, corrtype = "spearman")
```

## Vignette

The vignette can be found in the `vignettes/` directory. Further 
information is also available in the 'Instructions' tab of the shiny app. 
After installation, the vignette can be accessed from the R console by typing

```
browseVignettes("iCOBRA")
```

## Benchmarking data set collection

To facilitate future benchmarking studies, we have collected a set of benchmarking 
data sets on [http://imlspenticton.uzh.ch/robinson_lab/benchmark_collection/](http://imlspenticton.uzh.ch/robinson_lab/benchmark_collection/). The page provides 
links to raw data as well as evaluation results suitable for import into `iCOBRA`. 
