Skip to contents

16 Development

2026-03-21

Source: vignettes/articles/16_Development.Rmd
16_Development.Rmd

Here’s some stuff that’s in progress. These changes will show up first in the development dev branch, which you are welcome to install and test.

If you would like to contribute to AutoSpectral, these are key areas I have identified where it has shortcomings or needs improvement. Suggestions for strategies, or better, code suggestions, would be appreciated.

  • Unmixing with controls from different days or different voltages

  • AutoSpectral does not currently support this, although it may work reasonably well anyway since these modern spectral cytometers are fairly well calibrated for PMT/APD linearity.

  • In most cases, AutoSpectral does not have access to the information needed to normalize the files between different voltage settings. For Cytek instruments, this would be QC information and the initial linearity response curve for the APDs. For FACSDiscover systems, this is likely the QSPE values, which are in the FCS files.

  • I’d need to sit down and work out the normalization functions. This hasn’t been a top priority.

  • Parallelization improvements.

    • The parallelization of functions like define.flow.control was problematic on Windows due to memory usage with futures and the tendency of Windows to usurp threads that R was using.
    • A new parallel backend using parLapply on Windows or mclapply on Mac/Linux has been implemented. This should gracefully fall back to sequential lapply processing if it runs into any problems when being set-up.
    • Parallelization of unmix.autospectral() in base R can be ~30% faster for per-cell unmixing if we use foreach with doParallel, but this introduces at least three new dependencies and is a minor improvement compared with running via Rcpp.
    • I am struggling to produce fast adaptations of AutoSpectral for MacOS because I do not have access to MacOS. Suggestions would be appreciated for better implementation of the mclapply parallelization for Mac. Perhaps more importantly, AutoSpectralRcpp relies on OpenMP, and OpenMP is difficult to set up and use on Mac, in my understanding. If it would help, I could produce a version of AutoSpectralRcpp that just relies on RcppArmadillo without OpenMP.
    • Memory management. This is a problem for unmixing large files, I believe.

  • Code clean-up for define.flow.control.

    • Currently, we read in all of the FCS data into active memory in R, gating as we go to reduce data size. It might be better to restructure this to simply generate gates and retain indices of gated events in each file. FCS files would need to be read in on the fly for clean.controls and get.fluorophore.spectra. This would require a faster FCS reader, such as the data.table-based approach in Nathan Laniewski’s flowState.

  • More flexibility in negative/unstained samples.

  • Allow multiple controls per fluorophore.

  • An alternative to the automated gating.

    • This is now implemented in version 1.5.0
    • This is based on the “Cellular Landmarks” approach from Nathan Laniewski’s flowState

  • Speed up per-cell AF extraction

    • This is now implemented in v1.0.0, figuring out what each cell’s AF signature is without unmixing each AF possibility on all cells. Extraction is parallelized via C++ in AutoSpectralRcpp.

  • Speed up per-cell fluorophore optimization.

    • This is now implemented in v1.0.0 through strategic screening of variants for alignment with each cell’s profile/residual.
    • The pure R version will now benefit from parallelization, particularly on Mac/Linux.
    • At this point, I do not expect more major gains in performance. It has been suggested, however, that GPU acceleration may help. If you have experience with that and can offer insights, please reach out.

  • Fix the issue causing discontinuities.

    • Progress has been made on this in v1.0.0 via the new, sped-up optimization strategy. The “slow” method should now be fast enough to fix most of the issue.
    • Additional improvements include better decontamination of AF from the spectral variants, which was causing cells to be pushed further away from the threshold for optimization, and a rduction in low-level noise introduced into the spectral variants due to fluctuations in electronic noise or autofluorescence. Any signal in the variant spectrum in a channel where the optimized single spectrum has less than 5% of the peak detector signal is now regularized towards the optimized spectrum. This focuses the variation on the “peaks”, which is what causes the unmixing errors and spread.
    • This should be improved further with the next round of updates, tentatively 1.6.0

  • Better correction of unmixing errors

    • There are several strategies I’m investigating to handling the cases where the multi-color samples contain obvious errors outside the range of variation seen in the single-color controls. This is not stuff I’m going to put online, so get in touch if you’d like to work on this.
    • This should be improved further with the next round of updates, tentatively 1.6.0

To install the dev branch:

devtools::install_github("DrCytometer/AutoSpectral@dev")

To replace this with the (hopefully) stable version, run this:

remove.packages("AutoSpectral")
devtools::install_github("DrCytometer/AutoSpectral")