Dec 12, 2024

Public workspaceRetroClac: A Program for Harmonizing Reporting Paradigm in Diagnostic Accuracy Studies

DOI
dx.doi.org/10.17504/protocols.io.14egn961ql5d/v1
  • Mahdi Norouzi1
  • 1Isfahan University of Medical Sciences, Isfahan, Iran
Mahdi Norouzi
Icon indicating open access to content
QR code linking to this content
DOI: dx.doi.org/10.17504/protocols.io.14egn961ql5d/v1
External link: https://github.com/mdnx/RetroCalc
Protocol Citation: Mahdi Norouzi 2024. RetroClac: A Program for Harmonizing Reporting Paradigm in Diagnostic Accuracy Studies. protocols.io https://dx.doi.org/10.17504/protocols.io.14egn961ql5d/v1
License: This is an open access protocol distributed under the terms of the Creative Commons Attribution License,  which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Protocol status: Working
We use this protocol and it's working
Created: December 12, 2024
Last Modified: December 12, 2024
Protocol Integer ID: 115051
Keywords: Meta-analysis, Diagnostic test accuracy, clinical research, Diagnostic tests
Abstract
Accurate diagnostic test performance assessment is critical in clinical and research settings, where a wide range of metrics, including accuracy, sensitivity (recall), specificity, f1 score, positive predictive value (PPV), and negative predictive value (NPV), are used to evaluate test validity. However, heterogeneous reporting of these metrics across studies poses challenges for direct comparison and meta-analysis. RetroCalc, a novel program, addresses this gap by reconstructing essential confusion matrix values—true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN)—from minimal input pairs. RetroCalc enables accurate estimation of all relevant diagnostic metrics, even when only limited data is provided. This feature supports meta-analyses by harmonizing data inputs, facilitating more consistent and comprehensive reviews of diagnostic test accuracy studies. RetroCalc provides a valuable tool for enhancing the interpretability and comparability of diagnostic test outcomes, supporting a more unified and accessible approach to diagnostic accuracy research.
Attachments
Icon for RetroCalc.pdf
RetroCalc.pdf
258KB
Guidelines
Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy.
Input Preparation
Input Preparation
Prepare the total cohort sizes:
• Positive cohort (P = TP + FN)
• Negative cohort (N = TN + FP)
Identify one of the following pairs based on your data:


  1. Sensitivity (Sen) and Specificity (Spe)
  2. Sen and Positive Predictive Value (PPV)
  3. Sen and Negative Predictive Value (NPV)
  4. Sen and F1 Score (F1)
  5. Specificity (Spe) and Positive Predictive Value (PPV)
  6. Accuracy (Acc) and Sen
  7. Acc and Spe
  8. Spe and NPV
  9. Likelihood Ratios (LR+) and Spe
Install and Run RetroCalc
Install and Run RetroCalc
# clone the retrocalc into your own enviroment
!git clone https://github.com/mdnx/RetroCalc.git


%cd RetroCalc
!pip install .

# input the required metrics, remember not to use percentage (e.g. 45% incorrect, .45 correct)

from retrocalc.retrocalc import get_user_inputs


get_user_inputs()

RetroCalc Mechanism
RetroCalc Mechanism
 1. Launch the program from the directory:

 2. Provide the input metrics and cohort sizes when prompted.
4. Metric Validation & Reconstruction
 1. RetroCalc verifies the input metrics, avoiding division by zero or invalid values.
 2. It computes TP, TN, FP, and FN using predefined pairwise formulas.
 3. If the input pair is insufficient, the program suggests additional metrics for input.
5. Output and Visualization
 1. RetroCalc displays the reconstructed confusion matrix and recalculated diagnostic metrics.
 2. Visualize the matrix using the built-in plotting function for easier interpretation.
References
References
References
 • RetroCalc GitHub Repository: https://github.com/mdnx/RetroCalc
 • Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy.
Protocol references
Deeks JJ, Bossuyt PM. Chapter 3: Methods and presentation of systematic reviews of test accuracy. In: Deeks JJ, Bossuyt PM, Leeflang MM, Takwoingi Y editor(s). Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 2.0 (updated July 2023).