Intro

This report is the supplementary material for the AGS paper and has all the simulation results and investigations related to that paper, as well as instructions for reproducing the results.

Methodology/Input Overview

A list of things that change between the simulations and the presented figures are:

  • The number of Gitsbe simulations: more simulations, more models generated.
  • The type of mutation that Gitsbe models have: unless otherwise specified, the Gitsbe models have only link operator mutations. Topology mutations were also tested as well as a combination of topology and link operator mutations.
  • The training data for the Gitsbe models: steady state (calibrated models) vs proliferative profile (random models).
  • The type of mathematical model (HSA or Bliss) used in Drabme to evaluate the synergies either from the (Flobak et al. 2015) for the CASCADE 1.0 analysis or from the (Flobak et al. 2019) dataset for the CASCADE 2.0 analysis. More info on the calculations that Drabme does see here.
  • The type of output used from Drabme: ensemble-wise or model-wise synergy results.

Summary

Observing the results across the whole report, we reach the following conclusions:

  • To minimize the expected performance variance, executing \(150\) Gitsbe simulations (~\(500\) best-fitted models) is a good choice (no need for more, no matter the other input parameters).
  • Ensemble-wise results do not correlate with model-wise results (see correlation results for CASCADE 1.0 and CASCADE 2.0). This happens because some drug perturbed models do not have stable states and thus cannot be evaluated for synergy.1
  • Model-wise ROC results are always better compared to ensemble-wise ROC results for the single predictor models (e.g. the calibrated non-normalized model results).
  • When using a combined model predictor (see here) to augment/correct the calibrated models results, Drabme’s Bliss synergy assessment always brings significant performance benefit for the ensemble-wise results. When using HSA, that is not always the case (see one example and another).
  • The model-wise results do not bring any performance benefit when used in a combined predictor.
  • The value of \(\beta = -1\) is a good estimation for the value that maximizes the combined predictor’s performance (\(calibrated + \beta \times random\)) across all of the report’s relevant investigations.
  • Comparing the different parameterization schemes for the CASCADE 2.0 analysis (using the combined predictors with \(\beta = -1\)), we observe that topology mutations outperform link operator mutations.
  • There is correlation between fitness to the AGS steady state and normalized ensemble prediction performance. This is observed for the link operator mutated CASCADE 2.0 models here and a little bit more for the topology mutated ones. Same trend was shown for the CASCADE 1.0 link-operator mutated models analysis.
  • Any type of scrambling in the curated CASCADE topology reduces ensemble model prediction performance. See results for CASCADE 1.0 here and CASCADE 2.0 here.
  • Expression of ERK is a biomarker that distinguishes the higher performance AGS models (see results of the investigation here).

  1. Using minimal trapspaces, where there is almost always an attractor found and the global output of the model can be calculated, we observed higher correlation between ensemble-wise and model-wise results (as expected)↩︎