qsprpred.models.assessment.metrics package

Submodules

qsprpred.models.assessment.metrics.base module

class qsprpred.models.assessment.metrics.base.Metric[source]

Bases: ABC

Abstract class for scoring functions.

Variables:

name (str) – Name of the scoring function.

qsprpred.models.assessment.metrics.classification module

class qsprpred.models.assessment.metrics.classification.BEDROC(alpha: float = 20)[source]

Bases: Metric

Calculate the Boltzmann-enhanced discrimination of ROC (BEDROC).

Reference: Truchon and Bayly, J. Chem. Inf. Model. 2007 47 (2), 488-508. DOI: 10.1021/ci600426e

Variables:

name (str) – Name of the scoring function (bedroc).

Initialize the BEDROC scorer.

Parameters:

alpha (float) – Weighting parameter (default: 20)

class qsprpred.models.assessment.metrics.classification.BalancedCohenKappa[source]

Bases: Metric

Calculate the balanced Cohen kappa coefficient.

Guesné, S.J.J., Hanser, T., Werner, S. et al. Mind your prevalence!. J Cheminform 16, 43 (2024). https://doi.org/10.1186/s13321-024-00837-w

Variables:

name (str) – Name of the scoring function (balanced_cohen_kappa).

class qsprpred.models.assessment.metrics.classification.BalancedMatthewsCorrcoeff[source]

Bases: Metric

Calculate the balanced Matthews correlation coefficient.

Guesné, S.J.J., Hanser, T., Werner, S. et al. Mind your prevalence!. J Cheminform 16, 43 (2024). https://doi.org/10.1186/s13321-024-00837-w

Variables:

name (str) – Name of the scoring function (balanced_matthews_corrcoeff).

class qsprpred.models.assessment.metrics.classification.BalancedNegativePredictivity[source]

Bases: Metric

Calculate the balanced negative predictivity.

Guesné, S.J.J., Hanser, T., Werner, S. et al. Mind your prevalence!. J Cheminform 16, 43 (2024). https://doi.org/10.1186/s13321-024-00837-w

Variables:

name (str) – Name of the scoring function (balanced_negative_predictivity).

class qsprpred.models.assessment.metrics.classification.BalancedPositivePredictivity[source]

Bases: Metric

Calculate the balanced positive predictivity.

Guesné, S.J.J., Hanser, T., Werner, S. et al. Mind your prevalence!. J Cheminform 16, 43 (2024). https://doi.org/10.1186/s13321-024-00837-w

Variables:

name (str) – Name of the scoring function (balanced_positive_predictivity).

class qsprpred.models.assessment.metrics.classification.CalibrationError(n_bins: int = 10, norm: str = 'L1')[source]

Bases: Metric

Compute the calibration error of a classifier.

ECE is defined as the expected difference between the predicted probability and the observed frequency in each bin. The lower the ECE, the more calibrated the classifier is.

Referece: Guo et al. (2017) On Calibration of Modern Neural Networks. https://arxiv.org/abs/1706.04599

Variables:

name (str) – Name of the scoring function (calibration_error).

Initialize the calibration error scorer.

If norm is ‘L1’, the expected calibration error is returned (ECE). If norm is ‘L2’, the root-mean-square calibration error is returned (RMSCE). If norm is ‘infinity’, the maximum calibration error is returned (MCE).

Parameters:

  • n_bins (int) – Number of bins to use for calibration. A bigger bin number requires more data. Defaults to 10.

  • norm (str) – The norm to use for the calibration error. Can be ‘L1’ or ‘L2’ or ‘infinity’. Defaults to ‘L1’.

class qsprpred.models.assessment.metrics.classification.CohenKappa[source]

Bases: Metric

Calculate the Cohen’s kappa coefficient.

Variables:

name (str) – Name of the scoring function (cohen_kappa).

class qsprpred.models.assessment.metrics.classification.EnrichmentFactor(chi: float = 0.05)[source]

Bases: Metric

Calculate the enrichment factor.

Variables:

name (str) – Name of the scoring function (enrichment_factor).

Initialize the enrichment factor scorer.

Parameters:

chi (float) – Weighting parameter (default: 5%)

class qsprpred.models.assessment.metrics.classification.NegativePredictivity[source]

Bases: Metric

Calculate the negative predictivity.

Variables:

name (str) – Name of the scoring function (negative_predictivity).

class qsprpred.models.assessment.metrics.classification.PositivePredictivity[source]

Bases: Metric

Calculate the Positive predictivity.

Variables:

name (str) – Name of the scoring function (Positive_predictivity).

class qsprpred.models.assessment.metrics.classification.Prevalence[source]

Bases: Metric

Calculate the prevalence.

Variables:

name (str) – Name of the scoring function (prevalence).

class qsprpred.models.assessment.metrics.classification.RobustInitialEnhancement(alpha: float = 100)[source]

Bases: Metric

Calculate the robust initial enhancement.

Reference: Sheridan et al., J. Chem. Inf. Model. 2001 41 (5), 1395-1406. DOI: 10.1021/ci0100144

Variables:

name (str) – Name of the scoring function (robust_initial_enhancement).

Initialize the robust initial enhancement scorer.

Parameters:

alpha (float) – Weighting parameter (default: 100)

class qsprpred.models.assessment.metrics.classification.Sensitivity[source]

Bases: Metric

Calculate sensitivity (true positive rate).

Variables:

name (str) – Name of the scoring function (sensitivity).

class qsprpred.models.assessment.metrics.classification.Specificity[source]

Bases: Metric

Calculate specificity (true negative rate).

Variables:

name (str) – Name of the scoring function (specificity).

qsprpred.models.assessment.metrics.classification.derived_confusion_matrix(y_true: array, y_pred: array) → tuple[int, int, int][source]

Calculate the derived confusion matrix.

Parameters:

  • y_true (np.array) – Ground truth (correct) labels. 1d array.

  • y_pred (np.array) – Predicted labels. 2D array (n_samples, 1)

Returns:

The derived confusion matrix.

Prevalence, sensitivity and specificity.

Return type:

tuple[int, int, int]

qsprpred.models.assessment.metrics.masked module

class qsprpred.models.assessment.metrics.masked.MaskedMetric(metric: Metric)[source]

Bases: Metric

Wrapper for Metrics to handle missing target values.

Initialize the masked metric.

Parameters:

metric (Metric) – The metric to be masked.

qsprpred.models.assessment.metrics.regression module

class qsprpred.models.assessment.metrics.regression.AbsoluteAverageFoldError[source]

Bases: Metric

Calculate the absolute average fold error (AAFE).

The AAFE is also known as the geometric mean fold error (GMFE).

Variables:

name (str) – Name of the scoring function (absolute_average_fold_error).

class qsprpred.models.assessment.metrics.regression.AverageFoldError[source]

Bases: Metric

Calculate the average fold error (AFE).

Variables:

name (str) – Name of the scoring function (fold_error).

class qsprpred.models.assessment.metrics.regression.KPrimeSlope[source]

Bases: Metric

Calculate the slope of the regression line through the origin between the observed and predicted values.

Reference: Tropsha, A., & Golbraikh, A. (2010). In J.-L. Faulon & A. Bender (Eds.), Handbook of Chemoinformatics Algorithms. https://www.taylorfrancis.com/books/9781420082999

Variables:

name (str) – Name of the scoring function (k_prime_slope).

class qsprpred.models.assessment.metrics.regression.KSlope[source]

Bases: Metric

Calculate the slope of the regression line through the origin between the predicted and observed values.

Reference: Tropsha, A., & Golbraikh, A. (2010). In J.-L. Faulon & A. Bender (Eds.),

Handbook of Chemoinformatics Algorithms.

https://www.taylorfrancis.com/books/9781420082999

Variables:

name (str) – Name of the scoring function (k_slope).

class qsprpred.models.assessment.metrics.regression.Kendall[source]

Bases: Metric

Calculate the Kendall rank correlation coefficient.

Variables:

name (str) – Name of the scoring function (kendall).

class qsprpred.models.assessment.metrics.regression.Pearson[source]

Bases: Metric

Calculate the Pearson correlation coefficient.

Variables:

name (str) – Name of the scoring function (pearson).

class qsprpred.models.assessment.metrics.regression.PercentageWithinFoldError(fold_error: float = 2)[source]

Bases: Metric

Calculate the percentage of predictions within a certain fold error.

Variables:

name (str) – Name of the scoring function (percentage_within_{x}_fold_error).

Initialize the percentage within fold error scorer.

Parameters:

fold_error (float) – The fold error threshold. Defaults to 2.

class qsprpred.models.assessment.metrics.regression.R20[source]

Bases: KPrimeSlope

Calculate the coefficient of determination for regression line through the origin between the observed and predicted values.

Reference: Tropsha, A., & Golbraikh, A. (2010). In J.-L. Faulon & A. Bender (Eds.), Handbook of Chemoinformatics Algorithms. https://www.taylorfrancis.com/books/9781420082999

Variables:

name (str) – Name of the scoring function (r_2_0).

class qsprpred.models.assessment.metrics.regression.RPrime20[source]

Bases: KSlope

Calculate the coefficient of determination for regression line through the origin between the predicted and observed values.

Reference: Tropsha, A., & Golbraikh, A. (2010). In J.-L. Faulon & A. Bender (Eds.), Handbook of Chemoinformatics Algorithms. https://www.taylorfrancis.com/books/9781420082999

Variables:

name (str) – Name of the scoring function (r_prime_2_0).

class qsprpred.models.assessment.metrics.regression.Spearman[source]

Bases: Metric

Calculate the Spearman correlation

Variables:

name (str) – Name of the scoring function (spearman).

qsprpred.models.assessment.metrics.scikit_learn module

class qsprpred.models.assessment.metrics.scikit_learn.SklearnMetrics(scorer: str | _BaseScorer)[source]

Bases: Metric

Wrapper for sklearn scoring functions.

Variables:

  • name (str) – Name of the scoring function.

  • scorer – Sklearn scorer object.

Initialize the scoring function.

Parameters:

scorer (str | _BaseScorer) – Name of the scoring function or sklearn scorer

Module contents