The effect of model rescaling and normalization on sensitivity analysis on an example of a MAPK pathway model
- Jakob Kirch^{1},
- Caterina Thomaseth^{1},
- Antje Jensch^{1} and
- Nicole E. Radde^{1}Email author
DOI: 10.1140/epjnbp/s40366-016-0030-z
© Kirch et al. 2016
Received: 15 January 2016
Accepted: 7 April 2016
Published: 10 May 2016
Abstract
Background
The description of intracellular processes based on chemical reaction kinetics has become a standard approach in the last decades, and parameter estimation poses several challenges. Sensitivity analysis is a powerful tool in model development that can aid model calibration in various ways. Results can for example be used to simplify the model by elimination or fixation of parameters that have a negligible influence on relevant model outputs. However, models are usually subject to rescaling and normalization to reference experiments, which changes the variance of the output. Thus, the results of the sensitivity analysis may change depending on the choice of these rescaling factors and reference experiments. Although it might intuitively be clear, this fact has not been addressed in the literature so far.
Methods
In this study we investigate the effect of model rescaling and additional normalization to a reference experiment on the outcome of two different sensitivity analyses. Results are exemplified on a model for the MAPK pathway module in PC-12 cell lines. For this purpose we apply local sensitivity analysis and a global variance-based method based on Sobol sensitivity coefficients, and compare the results for differently scaled and normalized model versions.
Results
Results indicate that both sensitivity analyses are invariant under simple rescaling of variables and parameters with constant factors, provided that sensitivity coefficients are normalized and that the parameter space is appropriately chosen for Sobol’s method. By contrast, normalization to a reference experiment that also depends on parameters has a large impact on the results of any sensitivity analysis, and in particular complicates the interpretation.
Conclusion
This work shows that, in order to perform sensitivity analysis, it is necessary to take into account the dependency on parameters of the reference condition when working with normalized model versions.
Keywords
Global sensitivity analysis Sobol indices Model rescaling MAPK signaling pathwayBackground
Ordinary differential equations are the most commonly used mathematical approach to describe the dynamics of intracellular signaling pathways. They are often based on chemical reaction kinetics, and standard ways to describe reaction rates exist. Several models have been proposed for different signaling pathways, and analysis methods have been developed and applied for further model investigation. Many of these models are scaled and normalized to a predefined reference condition. Appropriate rescaling can simplify the model and for example remove non-identifiable parameters (see e.g. [1]). Normalization is often needed to compare models to any kind of normalized experimental data. A normalization of experimental data is generally required in all cases where measured outcomes do not allow to extract absolute amounts, but only values that are assumed to be proportional to these amounts. This is quite often the case, including for example Western blot or FACS data. Western blotting is a technique to quantify protein amounts and their activity states. Detection works via the quantification of light signals from antibodies that specifically bind to the protein under study. These light signals have to be normalized in a two-step procedure in order to enable a comparison between different replicates. In a first step, signals are normalized to a loading control, in order to minimize artifacts that are due to different loading amounts. Second, since raw signals depend on the specialties of the antibodies, the particular membranes and chemicals in use, they are usually additionally normalized to a reference condition (for examples see [2, 3]). This second normalization is required for a comparison of data from different replicates. Since Western blotting is also used more and more frequently for a quantitative analysis, several studies are involved with the experimental protocols, testing of linear ranges, and proper normalization procedures [4, 5]. In this work we use the term rescaling whenever model parameters and variables are multiplied by constant factors, which is often applied to obtain dimensionless models. In contrast, normalization refers to conditioning data to a reference experiment, though both terms are often used as synonyms in other references. From a modeling point of view, rescaling and normalization must sometimes be treated differently, since the reference experiment usually also depends on parameters, and in particular, is itself subject to variance.
Although normalized models are omnipresent all over, the effect of normalization on model calibration and analysis has not been well-investigated so far, and is also poorly understood. From a modeling point of view, the effect of normalization in a statistical framework for state estimation has been investigated [5–7]. Results indicate that normalization might also have a crucial effect on parameter estimation.
Here we consider the effect of rescaling and normalization on sensitivity analysis. Sensitivity analysis is one of the most important tools in model development and can for example be used for model reduction, calibration, validation, robustness analysis or the design of experiments. This type of model analysis is widely applicable in various scientific fields such as engineering, physics, economy, social sciences and many more. Some nice examples from different applications are illustrated in [8]. Various mathematical definitions of sensitivity functions exist, with different methods for their computation. However, the common basic idea is to quantify the variation of an output of a mathematical model due to variations of some input quantity, such as for example a model parameter or an initial condition. In some cases, the output of interest is a time-invariant function of the input, for example the steady state of the system. When considering dynamical systems, described for example with ODE models, the output of interest is generally the whole trajectory of the state variable, rendering the analysis more challenging. An introduction into sensitivity theory for continuous- and discrete-time dynamical systems is provided in [9], with a particular focus on linear systems. This book is mainly interesting for control engineering applications, since it defines sensitivity functions in time and frequency domains, and investigates optimal control systems. Zi et al. [10] and Kim et al. [11] instead provide good reviews about the application of different sensitivity analysis methods in systems biology, including advices for toolboxes and implementations and its role for model development. Further application of global sensitivity methods on different biologically inspired example models are described in [12] and [13], the former with a special focus on computational efficiency. Kent et al. [13] define a sensitivity based robustness measure, which is evaluated on five different models, including also a model for the steady states in the MAPK signaling module.
In this study, we exemplarily investigate the impact of rescaling and normalization to a reference experiment on sensitivity analysis for a toy model describing a simple reversible reaction to illustrate results and, as a real world case study, a model of the MAPK signaling pathway module in PC-12 cell lines. For the latter we use a model that was calibrated to Western blot data from an experimental study in Santos et al. [14]. Parameter estimation was done via a sampling-based Bayesian approach. For this study we use the maximum-a-posteriori (MAP) estimator as a point estimate. We compare results of local and global, variance-based sensitivity analysis methods on three model versions. The native model version A describes the dynamics of activities of proteins that take part in the signaling cascade. In model B all variables are rescaled to their respective total protein amounts, which are assumed to be conserved. Thus variables represent fractions of total amounts. Finally, in order to compare this model to Western blot data, the model output was additionally normalized to a reference condition, which defines model variant C. This last model version was the one used in [15] for parameter estimation. We compare the results of local and global, variance-based sensitivity analysis on these three model variants. We decided to use the Sobol sensitivity analysis method [16, 17], since it is one of the most general methods that, different from other methods, does not rely on monotone or even linear input/output relationships [10]. Moreover, Sobol sensitivity indices have been shown to highly correlate with other sensitivity measures, such as indices from Extended Fourier Amplitude Sensitivity Tests (FAST) and Partial Rank Correlation Coefficients (PRCC), indicating a kind of robustness of these measures [18]. They are furthermore recently pointed out as advantageous in other respects as well in connection with pharmacology models [19].
The paper is structured as follows. We start deriving general results for the effect of rescaling and normalization on local and global sensitivity analysis. These are compared and discussed for all three model versions of the toy model, which partly allows to illustrate effects by analytic calculations. Then we introduce the ODE modeling approach for the MAPK signaling pathway module and discuss numerically obtained results for this case study. Details of the sensitivity methods can be found in the Methods section.
Results and discussion
The effect of rescaling and normalization on sensitivity analysis
Local sensitivity analysis
which equals the normalized sensitivity coefficient s(t) of model version 1.
Thus, the local sensitivity coefficients s ^{′}(t) are shifted by the respective local sensitivity s(t ^{∗}) of the reference experiment. Hence all sensitivity coefficients become zero at the reference conditions, i.e. s ij′(t ^{∗})=0.
In summary, normalized local sensitivity coefficients are invariant under rescaling of model variables and parameters. Additional normalization to a parameter dependent reference experiment shifts the sensitivity courses by the local sensitivity coefficient of the reference experiment. Hence the sensitivity coefficient becomes zero at the reference experiment. Positive values \(s^{\prime }_{ij}(t)\) indicate that the relative change of the respective concentration exceeds that of the reference experiment, while negative values indicate that the relative change in the reference experiment is larger. In order to interpret these results in terms of the total concentrations, one has to take the sensitivity value of the reference experiment into account.
Variance-based global sensitivity analysis
Variance-based sensitivity analysis decomposes the variance of the output Y due to variations in the input parameters into contributions from different inputs. Here we exploit the Sobol sensitivity analysis method, shortly Sobol method, which can be applied to any non-linear differential equation model. Generally, this method decomposes the variance of each output into a sum of 2^{ k }−1 terms, k denoting the number of influential parameters, that describe the contribution of each possible parameter subgroup to the total variance. A more detailed mathematical description of this method is provided in [16, 17] and is recapitulated in the Methods section. Computations can drastically be reduced by only considering the so-called first order and total effect sensitivity indices S _{ i } and \(S_{T_{i}}\), i=1,…,k. The first order indices quantify the contribution of variations in parameter P _{ i } only to the total output variance, while \(S_{T_{i}}\), on the other side of the spectrum, is the overall effect of parameter P _{ i }, in contribution with variations of all possible combinations of the other parameters. Thus, S _{ Ti }≥S _{ i }, and the difference quantifies the interaction of parameter P _{ i } with the other model parameters. Furthermore, S _{ Ti },S _{ i }∈[0,1], and S _{ Ti }=0 implies that P _{ i } has no effect at all on the output, while S _{ Ti }=1 indicates that the output variance can completely be assigned to the variance in the factor P _{ i }.
Thus, \(S^{\prime }_{i}\) and \(S^{\prime }_{Ti}\) contain expectation values and variances of ratios of random variables, which cannot further be resolved in the general case. Hence, such a normalization to a reference experiment might generally change the outcome of this sensitivity analysis completely. Furthermore, since ratio distributions can be difficult to handle, and in particular, moments might even not be defined at all [20, 21], a reference experiment normalization can considerably complicate this kind of sensitivity analysis. At least, convergence has to be checked carefully for a Monte Carlo implementation of the method.
Case study I: sensitivity analysis for a simple reversible reaction
For illustration purposes, we first consider the effect of rescaling and normalization on a simple reversible reaction,
Case study I: local sensitivity analysis
Case study I: global sensitivity analysis
which gives \(s_{k_{1}}^{A}\approx 0.4675\).
Sobol sensitivity indices are the same for model version B, provided that \({k_{1}^{B}}={k_{1}}/{{x_{0}^{A}}}\) is sampled from \({k_{1}^{B}}\sim U\left (1/{x_{0}^{A}},2/{x_{0}^{A}}\right)\).
Figure 3 shows that the shape of the density \(f_{Y^{C}}\left (y^{C}\right)\) is different from \(f_{Y^{A}}\left (y^{A}\right)\) (bottom left). Furthermore, as can be seen from the Sobol indices, most of the variance of Y ^{ C } is attributed to variations in the parameter \({k_{2}^{B}}\), while the parameter \({k_{1}^{B}}\) has only a marginal influence. Also the interaction effect between both parameters is not very large. These results are reflected in the Figures on the right hand side: The remaining variance in Y ^{ C } when fixing \({k_{1}^{B}}\) at a certain value (top right) is much higher than the respective variance when fixing \({k_{2}^{B}}\) (bottom right), and this is true for all possible values of \({k_{1}^{B}}\) and \({k_{2}^{B}}\). Moreover, while the mean value \(\mathbb {E}_{Y^{C}|k_{1}^{B,*}}\left (Y^{C}\right)\) does hardly change as a function of \(k_{1}^{B,*}\), \(\mathbb {E}_{Y^{C}|k_{2}^{B,*}}\left (Y^{C}\right)\) highly varies as a function of \(k_{2}^{B,*}\), resulting in a small first order Sobol index \(s_{{k_{1}^{B}}}^{C}\) and a large Sobol index \(s_{{k_{2}^{B}}}^{C}\).
Overall, results on this simple toy model illustrate the effect of rescaling and normalization on sensitivity analysis.
Case study II: a model for the MAPK module in PC-12 cell lines
The Mitogen-activated protein kinase (MAPK) cascade is a conserved signaling module that is part of various signaling pathways. It is a three-tired phosphorylation cascade, which involves the proteins Raf, MEK and ERK. Raf is activated by Ras upon stimulation, which then triggers the double phosphorylation of MEK. Phosphorylated MEK in turn phosphorylates and thereby activates ERK, which also requires double phosphorylation to become fully active. ERK has a lot of substrates that regulate different cellular fates. The MAPK pathway is a well investigated signaling module from an experimental and a modeling point of view [22–24]. It can show a rich variety of different behaviors such as oscillations, ultrasensitivity, or bistability and has been investigated in different contexts.
Specificity in the response of the MAPK module to different ligands, which ensures a reliable processing of signals, is achieved through different courses in ERK activity, which in turn regulate ERK substrate activation. In particular, the MAPK module involves several feedback regulations, which are important to shape ERK response. Most importantly, ERK interacts with Raf via different mechanisms in a context dependent manner. This has been exemplified in a study with PC-12 cell lines, in which the MAPK signaling pathway was investigated upon stimulation with epidermal growth factor (EGF) and neuronal growth factor (NGF) [14]. PC-12 cells show a transient ERK activity after stimulation with EGF, and cells start to proliferate. In contrast, ERK activity is sustained for at least one hour after stimulation with NGF, and NGF triggers differentiation.
Model variant A is an unnormalized version, whose state variables correspond to the actual amounts of these four proteins. The ODE model corresponding to model version A is shown in Fig. 4 b.
Reaction rate constants are denoted by \(k^{(+/-)}_{i}\) and \(\tilde k^{(+/-)}_{i}\). The input u(t) mimics transient Ras activation upon stimulation and is described by a sigmoidally decreasing function. The positive feedback from ERK to Raf is described by a Hill-type function. The Hill coefficients were set to m=5 and M=3.
where \(\tilde p\) denotes the vector of parameters of the system.
where p denotes the vector of rescaled parameters, obtained from \(\tilde {p}\).
MAP parameter values used for the sensitivity analysis
θ | \(\log k_{1}^{+}\) | \(\log k_{2}^{+}\) | \(\log k_{3}^{+}\) | \(\log k_{4}^{+}\) | \(\log k_{1}^{-}\) | \(\log k_{2}^{-}\) | \(\log k_{3}^{-}\) | \(\log k_{4}^{-}\) | logk _{ Fp } | logg | K |
---|---|---|---|---|---|---|---|---|---|---|---|
\(\hat {\theta }^{\text {MAP}}\) | –5.7324 | 7.3475 | 7.8110 | 2.3365 | –0.0865 | 6.2055 | 6.8132 | –0.4295 | -5.9037 | –5.8563 | 5.6202 |
Case study II: local sensitivity analysis
Overall, the results of this local sensitivity analysis are plausible given the model structure. The results in particular show that the early transient behavior of the cascade is mainly determined by the phosphorylation rates. Moreover, the time point at which trajectories return to their steady state is very sensitive to changes in most parameters, \(k_{1}^{+}\) and K being the only exceptions.
Case study II: Variance-based global sensitivity analysis
Looking at the first order indices, the number of influential factors increases from Raf to MEK, which is naturally expected, since the signal mainly propagates in this direction. Consequently, pRaf is mainly influenced by the rate constants of its own dephosphorylation, followed by it’s phosphorylation rate \(k_{1}^{+}\), which is more dominant in the beginning of the response, where it determines the speed of Raf activation. Raf activity is furthermore weakly dependent on the feedback strength k _{ Fp }. In addition, the phosphorylation and dephosphorylation rates of MEK come into play in the first order Sobol sensitivities of MEK. Finally, the course of ppERK is influenced by a mixture of rate constants of Raf, MEK and ERK phosphorylation and dephosphorylation.
The values of the sums of total effect indices, which are shown in the second row, are much larger than the first order indices, indicating that interaction effects have an important impact on the overall variance of the model outputs. Compared to the first order effects, some new parameters appear, such as the parameter K, which had little effect as first order indices. This indicates that K influences the model output mainly via interactions with other parameters. While Raf is still mainly influenced by its dephosphorylation rate, the number of influential parameters increases from Raf to MEK, and ERK activity is regulated by a mixture of various total effect indices.
Since pERK is a hidden variable that is not observed, its phosphorylation and dephosphorylation rates \(k_{3}^{+}\) and \(k_{3}^{-}\) have only a marginal influence on model outputs. This is probably also due to the fact that pERK is an intermediate product between inactive non-phosphorylated and fully active, double phosphorylated ERK, which often has a buffering role and makes the overall system less sensitive to changes in e.g. total protein concentrations.
It can further be seen that the first order indices rapidly decrease over time for all three observables, and are nearly zero at t=60 min after stimulation. The first order sensitivities for Raf and MEK are almost indistinguishable, which probably comes from the fact that both components also have very similar time courses after normalization. Similar to model version A, all three components are highly dominated by the dephosphorylation rate of Raf. Different from model version A, the feedback parameters k _{ Fp }, g, M and K become more prominent in the course of the first order indices especially for Raf and MEK. This is true for the first order and the total effect indices, and presumably comes from the fact that it regulates to a certain extend the time and the height of the maxima of all components, and therefore causes variances in the experiment used for normalization. The first order indices are all rather small for ERK over the entire time course.
The sum of total order effects are much larger than in model variant A for all time points and all components, showing that normalization indeed increases interaction effects among parameters. Moreover, all three components show a well balanced mixture of total order effects of all model parameters, suggesting that all normalized components are highly interconnected. Interpretation of Sobol indices of model version C and their meaning for the biological system is generally difficult.
Conclusions
We have demonstrated that normalized sensitivity coefficients and Sobol indices are invariant under simple rescaling of model variables and parameters. This is, however, different for a normalization to a reference experiment, whose value depends itself on model parameters. Such a normalization may change the results of both local and global sensitivity analysis completely. This has to be taken into account when working with relative data that are normalized to a reference experiment and models that are normalized in the same way to reproduce these relative data. Interpretation of the sensitivity coefficients or Sobol indices can be very difficult in this case. In particular, it is generally not possible to extract any information about respective changes of the unnormalized model trajectories. A sensitivity coefficient near zero, for example, just indicates that the relative changes of the reference experiment value and the respective considered model output are of the same order of magnitude. Thus, parameters that have a large impact on model outputs and appear to be important in an unnormalized model version, might have small sensitivity values in a normalized model version, and vice versa.
Dealing with relative data and corresponding normalized models generally poses a challenge from the modeling point of view. As shown in this work, it renders the interpretation of sensitivity analysis and its meaning for the biological system a difficult task. Furthermore, related to this issue, it complicates model inference and in particular parameter estimation. Parameter estimation is often formulated as an optimization problem with an objective function that comprises a comparison of the relative experimental data with the respective model predictions. Evaluation of this objective function requires two simulations for one experimental value, the reference experiment value, which is used for normalization, and the actual experiment. Hence although the objective function is independent of any scaling factors, factors of proportionality α have to be chosen for the individual simulations, which often causes numerical problems when not chosen properly. We also encountered such numerical instabilities in our sensitivity analyses, which requires some care in the choice of these factors.
In conclusion, the calibration and analysis of normalized models is challenging, and proper normalization methods and their impact on the analysis results remain an issue for further studies.
Methods
Local sensitivity analysis and the direct differential method
This differential equation system for S _{ j } can be solved numerically. It involves the Jacobian matrix J _{ f } of the system, which has to be defined and implemented. The direct differential method does not rely on the choice of an appropriate Δ p, but can be very time consuming especially for larger systems.
Variance-based sensitivity analysis
The main idea of variance-based sensitivity analysis methods is to decompose the variance of a model outcome according to the input factors. Variance-based methods are global methods, since they exploit the impact of parameters within a whole range of values. Moreover, in contrast to local sensitivity factors, they allow for the investigation of interaction effects between groups of parameters. Sobol indices are sensitivity measures that are based on average partial variances.
with expectation value \(\mathbb {E}(Y)\) and variance Var(Y). Importantly, all X _{ i } are assumed to be independent for the following procedure, and hence can be drawn independently from their marginal distributions.
where \(\operatorname {Var}_{ij}^{\,c}\) measures the joint effect of X _{ i } and X _{ j } on the output Y. S _{ ij } is denoted second-order index. Higher-order indexes can be derived accordingly.
which contains all terms of any order that include x _{ i }.
The Russian mathematician I.M. Sobol proposed a straightforward Monte Carlo-based estimation procedure for the first and total order sensitivity indices for the special case that the X _{ i } are sampled from the standard uniform distribution U(0,1).
with Ω _{ i } and Ω _{ j } denoting subsets of the index set {1,…,n}.
Varij is denoted second-order effect. Using the expansion in Eq. (46) to calculate the variance Var(Y), and exploiting that the means of the individual summands vanish and that the terms are orthogonal, we get the following decomposition
This decomposition gives rise to define the total effect index \(S_{T_{i}}\) of a component X _{ i } as the total effect of X _{ i } on Y, which is the sum of all sensitivity indices containing X _{ i }. S _{ Ti }=0 implies that X _{ i } does not influence Y at all and hence X _{ i } could for instance be set to a fixed value for further analysis.
since \(\mathbb {E}_{\sim X_{i}}(\operatorname {Var}_{X_{i}}(Y|X_{\sim i}))\) is the average variance after fixing all but variable X _{ i }.
The numerical procedure that is used to estimate S _{ i } and \(S_{T_{i}}\) is described in [16] and in [26] (a more recent study with a focus on implementation is [27]) and uses Monte Carlo integration to evaluate the integrals.
Declarations
Acknowledgements
This work was supported by the German Federal Ministry of Education and Research (BMBF) within the e:Bio-Innovationswettbewerb Systembiologie project PREDICT (grant number FKZ0316186A) and the German Research Foundation (DFG) within the Cluster of Excellence in Simulation Technology (EXC 310/1) at the University of Stuttgart.
licensee Springer on behalf of EPJ. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Authors’ Affiliations
References
- Segel LA. Modeling Dynamic Phenomena in Molecular and Cellular Biology. Cambridge: Cambridge University Press; 1984.MATHGoogle Scholar
- Möller Y, Siegemund M, Beyes S, Herr R, Lecis D, Delia D, Kontermann R, Brummer T, Pfizenmaier K, Olayioye M. EGFR-targeted TRAIL and a Smac mimetic synergize to overcome apoptosis resistance in KRAS mutant colorectal cancer cells. PLoS ONE. 2014; 9(9):e107165.ADSView ArticleGoogle Scholar
- Zinöcker S, Vaage J. Rat mesenchymal stromal cells inhibit T cell proliferation but not cytokine production through inducible nitric oxide synthase. Front Immunol. 2012; 3(62):1–13.Google Scholar
- Taylor SC, Posch A. The design of a quantitative Western blot experiment. BioMed Res Int. 2014; 2014:8. Article ID 361590.View ArticleGoogle Scholar
- Degasperi A, Birtwistle MR, Volinsky N, Rauch J, Kolch W, Kholodenko BN. Evaluating strategies to normalize biological replicates of Western Blot data. PLoS ONE. 2014; 9(1):e87293.ADSView ArticleGoogle Scholar
- Kreutz C, Rodriguez MMB, Maiwald T, Seidl M, Blum HE, Mohr L, Timmer J. An error model for protein quantification. Bioinformatics. 2007; 23(20):2747–53.View ArticleGoogle Scholar
- Thomaseth C, Radde N. Normalization of Western blot data affects the statistics of estimators. 2016. Submitted to FOSBE.Google Scholar
- Saltelli A, Tarantola S, Campolongo F, Ratto M. Sensitivity Analysis in Practice: a Guide to Assessing Scientific Models. Chichester: John Wiley & Sons; 2004.MATHGoogle Scholar
- Frank PM. Introduction to System Sensitivity Theory. New York: Academic Press Inc; 1978.MATHGoogle Scholar
- Zi Z. Sensitivity analysis approaches applied to systems biology models. IET Syst Biol. 2011; 5(6):336–46.View ArticleGoogle Scholar
- Kim KA, Spencer SL, Albeck JG, Burke JM, Sorger PK, Gaudet S, Kim DH. Systematic calibration of a cell signaling network model. BMC Bioinf. 2010; 11(202):1–14.Google Scholar
- Kiparissides A, Kucherernko SS, Mantalaris A, Pistikopoulus EN. Global sensitivity analysis challenges in biological systems modeling. Ind Eng Chem Res. 2009; 48(15):7168–80.View ArticleGoogle Scholar
- Kent E, Neumann S, Kummer U, Mendes P. What can we learn from global sensitivity analysis of biochemical systems?PLoS ONE. 2013; 8(11):e79244.ADSView ArticleGoogle Scholar
- Santos SDM, Verveer PJ, Bastiaens PIH. Growth factor-induced MAPK network topology shapes Erk response determining PC-12 cell fate. Nat Cell Biol. 2007; 9(3):324–30.View ArticleGoogle Scholar
- Jensch A, Thomaseth C, Radde N. Sampling-based Bayesian approaches reveal the importance of quasi-bistable behavior in cellular decision making processes on the example of the MAPK signaling pathway in PC-12 cell lines. Under review. 2016. BMC Systems Biology (Under review).Google Scholar
- Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D, Saisana M, Tarantola S. Global Sensitivity Analysis: The Primer. Hoboken, NJ: John Wiley & Sons; 2008.MATHGoogle Scholar
- Sobol IM. Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math Comput Simul. 2001; 55:271–80.MathSciNetView ArticleMATHGoogle Scholar
- Zheng Y, Rundell A. Comparative study of parameter sensitivity analyses of the TCR-activated ERK-MAPK signalling pathway. IEE Proc Syst Biol. 2006; 153(4):201–11.View ArticleGoogle Scholar
- Zhang XY, Trame MN, Lesko LJ, Schmidt S. Sobol sensitivity analysis: a tool to guide the development and evaluation of systems pharmacology models. CPT Pharmacometrics Syst Pharmacol. 2015; 4:69–79.View ArticleGoogle Scholar
- Hinkley DV. On the ratio of two correlated normal random variables. Biometrika. 1969; 56(3):635–9.MathSciNetView ArticleMATHGoogle Scholar
- Marsaglia G. Ratios of normal variables. J Stat Softw. 2006; 16(4):1–10.View ArticleGoogle Scholar
- Kholodenko B. Negative feedback and ultrasensitivity can bring about oscillations in the mitogen-activated protein kinase cascade. Eur J Biochem. 2000; 267(6):1583–88.View ArticleGoogle Scholar
- Kolch W. Coordinating ERK/MAPK signalling through scaffolds and inhibitors. Nat Rev Mol Cell Biol. 2005; 6:827–37.View ArticleGoogle Scholar
- Kolch W, Calder M, Gilbert D. When kinases meet mathematics: the systems biology of MAPK signalling. FEBS Lett. 2005; 579(8):1891–95.View ArticleGoogle Scholar
- Gouzé JL. Positive and negative circuits in dynamical systems. J Biol Syst. 1998; 6(11):11–15.View ArticleMATHGoogle Scholar
- Saltelli A, Annoni P, Azzini I, Campolongo F, Ratto M, Tarantola S. Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Comp Phys Comm. 2010; 181:259–70.ADSMathSciNetView ArticleMATHGoogle Scholar
- Bilal N. Implementation of Sobol’s method of global sensitivity analysis to a compressor simulation model. In: 22nd Int. Compressor Eng. Conf. Purdue, USA: Purdue e-Pubs: 2014.Google Scholar