% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/converters.R
\name{bigSpectronauttoMSstatsFormat}
\alias{bigSpectronauttoMSstatsFormat}
\title{Convert out-of-memory Spectronaut files to MSstats format.}
\usage{
bigSpectronauttoMSstatsFormat(
  input_file,
  output_file_name,
  backend,
  intensity = "F.NormalizedPeakArea",
  filter_by_excluded = FALSE,
  filter_by_identified = FALSE,
  filter_by_qvalue = FALSE,
  qvalue_cutoff = 0.01,
  max_feature_count = 100,
  filter_unique_peptides = FALSE,
  aggregate_psms = FALSE,
  filter_few_obs = FALSE,
  remove_annotation = FALSE,
  calculateAnomalyScores = FALSE,
  anomalyModelFeatures = c(),
  annotation = NULL,
  connection = NULL,
  block_size = 16L * 1024L * 1024L
)
}
\arguments{
\item{input_file}{name of the input text file in 10-column MSstats format.}

\item{output_file_name}{name of an output file which will be saved after pre-processing}

\item{backend}{"arrow" or "sparklyr". Option "sparklyr" requires a spark installation
and connection to spark instance provided in the `connection` parameter.}

\item{intensity}{Name of the intensity column to be used in Spectronaut}

\item{filter_by_excluded}{if TRUE, will filter by the `F.ExcludedFromQuantification` column.}

\item{filter_by_identified}{if TRUE, will filter by the `EG.Identified` column.}

\item{filter_by_qvalue}{if TRUE, will filter by EG.Qvalue and PG.Qvalue columns.}

\item{qvalue_cutoff}{cutoff which will be used for q-value filtering.}

\item{max_feature_count}{maximum number of features per protein. Features will
be selected based on highest average intensity.}

\item{filter_unique_peptides}{If TRUE, shared peptides will be removed.
Please refer to the `Details` section for additional information.}

\item{aggregate_psms}{If TRUE, multiple measurements per PSM in a Run will
be aggregated (by taking maximum value). Please refer to the `Details` section for additional information.}

\item{filter_few_obs}{If TRUE, feature with less than 3 observations across runs will be removed.
Please refer to the `Details` section for additional information.}

\item{remove_annotation}{If TRUE, columns BioReplicate and Condition will be removed
to reduce output file size. These will need to be added manually later before
using dataProcess function. Only applicable to sparklyr backend.}

\item{calculateAnomalyScores}{If TRUE, will carry anomaly model features through pipeline}

\item{anomalyModelFeatures}{Character vector of column names to be carried through the pipeline}

\item{annotation}{Optional data.frame with columns `Run`,
`BioReplicate`, `Condition` (plus any additional annotation
columns). If supplied, the converter merges it onto the output
on `Run` and overrides any `Condition` / `BioReplicate` values
that came from Spectronaut's `R.Condition` / `R.Replicate`
columns. Required when the experimental design cannot be
expressed in Spectronaut's own annotation — most notably paired
designs, where `BioReplicate` must encode the pairing
structure rather than per-sample IDs.}

\item{connection}{Connection to a spark instance created with the
`spark_connect` function from `sparklyr` package.}

\item{block_size}{Arrow CSV reader block size in bytes: the amount of raw
CSV that is read and parsed at a time (roughly one record batch per block).
Defaults to 16 MiB (`16L * 1024L * 1024L`). Two things matter when tuning it:
\itemize{
  \item It must be larger than your single widest row. If you see
    `Invalid: straddling object straddles two block boundaries` on
    extra-wide exports, raise it (e.g. `64L * 1024L * 1024L`).
  \item Larger blocks reduce parsing overhead, but peak memory grows
    roughly linearly with `block_size`, so it is also the main lever on
    out-of-memory risk. Gains taper off quickly because per-batch time is
    dominated by row cleaning, not by block size.
}
To speed up processing without risking a crash, double the value
(32 -> 64 -> 128 MiB) while watching memory use, and stop once throughput
stops improving. Keep it comfortably below free RAM -- and below free RAM
divided by the number of jobs if you run several conversions in parallel.}
}
\value{
either arrow object or sparklyr table that can be optionally collected
into memory by using dplyr::collect function.
}
\description{
Convert out-of-memory Spectronaut files to MSstats format.
}
\details{
`bigSpectronauttoMSstatsFormat` consumes the following Spectronaut export
columns.

Always required:
\itemize{
  \item `R.FileName`, `R.Condition`, `R.Replicate` -> Run, Condition, BioReplicate
  \item `PG.ProteinAccessions` -> ProteinName
  \item `EG.ModifiedSequence` -> PeptideSequence
  \item `FG.Charge` -> PrecursorCharge; `F.FrgIon` -> FragmentIon; `F.Charge` -> ProductCharge
  \item the intensity column (`F.NormalizedPeakArea` by default; see `intensity`) -> Intensity
  \item `F.FrgLossType` -> used by the always-on `noloss` fragment filter
}


Required only when the matching filter is enabled:
\itemize{
  \item `EG.Identified` -> `filter_by_identified`
  \item `F.ExcludedFromQuantification` -> `filter_by_excluded`
  \item `EG.Qvalue` and `PG.Qvalue` -> `filter_by_qvalue`
}
A missing required column raises an informative error naming the column.
}
\examples{
converted_data <- bigSpectronauttoMSstatsFormat(
  system.file("extdata", "spectronaut_input.csv", package = "MSstatsBig"),
  "output_file.csv",
  backend = "arrow")
converted_data <- dplyr::collect(converted_data)
head(converted_data)

# Override Spectronaut's embedded Condition / BioReplicate with
# a custom annotation (e.g. for a paired design):
annot <- data.frame(Run = unique(converted_data[["Run"]]))
annot$BioReplicate <- seq_len(nrow(annot))
annot$Condition <- rep(c("ctrl", "treat"), length.out = nrow(annot))
overridden <- bigSpectronauttoMSstatsFormat(
  system.file("extdata", "spectronaut_input.csv", package = "MSstatsBig"),
  "output_file.csv",
  backend = "arrow",
  annotation = annot)
head(dplyr::collect(overridden))

}
