Dec 12, 2024

Public workspaceRetroClac: A Program for Harmonizing Reporting Paradigm in Diagnostic Accuracy Studies

DOI
https://dx.doi.org/10.17504/protocols.io.14egn961ql5d/v1
  • Mahdi Norouzi1
  • 1Isfahan University of Medical Sciences, Isfahan, Iran
Mahdi Norouzi
Icon indicating open access to content
QR code linking to this content
DOI: https://dx.doi.org/10.17504/protocols.io.14egn961ql5d/v1
External link: https://github.com/mdnx/RetroCalc
Protocol Citation: Mahdi Norouzi 2024. RetroClac: A Program for Harmonizing Reporting Paradigm in Diagnostic Accuracy Studies. protocols.io https://dx.doi.org/10.17504/protocols.io.14egn961ql5d/v1
License: This is an open access protocol distributed under the terms of the Creative Commons Attribution License,  which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Protocol status: Working
We use this protocol and it's working
Created: December 11, 2024
Last Modified: December 12, 2024
Protocol Integer ID: 115051
Keywords: Meta-analysis, Diagnostic test accuracy, clinical research, Diagnostic tests, diagnostic test accuracy study, diagnostic accuracy studies accurate diagnostic test performance assessment, diagnostic accuracy studies accurate, diagnostic accuracy research, diagnostic test outcome, comparability of diagnostic test outcome, diagnostic test performance assessment, accessible approach to diagnostic accuracy research, relevant diagnostic metric, essential confusion matrix value, false positive, program for harmonizing reporting paradigm, negative predictive value, harmonizing reporting paradigm, test validity, false negative, including accuracy, heterogeneous reporting of these metric, f1 score, heterogeneous reporting, retrocalc, retroclac, recall
Abstract
Accurate diagnostic test performance assessment is critical in clinical and research settings, where a wide range of metrics, including accuracy, sensitivity (recall), specificity, f1 score, positive predictive value (PPV), and negative predictive value (NPV), are used to evaluate test validity. However, heterogeneous reporting of these metrics across studies poses challenges for direct comparison and meta-analysis. RetroCalc, a novel program, addresses this gap by reconstructing essential confusion matrix values—true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN)—from minimal input pairs. RetroCalc enables accurate estimation of all relevant diagnostic metrics, even when only limited data is provided. This feature supports meta-analyses by harmonizing data inputs, facilitating more consistent and comprehensive reviews of diagnostic test accuracy studies. RetroCalc provides a valuable tool for enhancing the interpretability and comparability of diagnostic test outcomes, supporting a more unified and accessible approach to diagnostic accuracy research.
Attachments
Icon for RetroCalc.pdf
RetroCalc.pdf
258KB
Guidelines
Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy.
Troubleshooting
Input Preparation
Prepare the total cohort sizes:
• Positive cohort (P = TP + FN)
• Negative cohort (N = TN + FP)
Identify one of the following pairs based on your data:


  1. Sensitivity (Sen) and Specificity (Spe)
  2. Sen and Positive Predictive Value (PPV)
  3. Sen and Negative Predictive Value (NPV)
  4. Sen and F1 Score (F1)
  5. Specificity (Spe) and Positive Predictive Value (PPV)
  6. Accuracy (Acc) and Sen
  7. Acc and Spe
  8. Spe and NPV
  9. Likelihood Ratios (LR+) and Spe
Install and Run RetroCalc
# clone the retrocalc into your own enviroment
!git clone https://github.com/mdnx/RetroCalc.git


%cd RetroCalc
!pip install .

# input the required metrics, remember not to use percentage (e.g. 45% incorrect, .45 correct)

from retrocalc.retrocalc import get_user_inputs


get_user_inputs()

RetroCalc Mechanism
 1. Launch the program from the directory:

 2. Provide the input metrics and cohort sizes when prompted.
4. Metric Validation & Reconstruction
 1. RetroCalc verifies the input metrics, avoiding division by zero or invalid values.
 2. It computes TP, TN, FP, and FN using predefined pairwise formulas.
 3. If the input pair is insufficient, the program suggests additional metrics for input.
5. Output and Visualization
 1. RetroCalc displays the reconstructed confusion matrix and recalculated diagnostic metrics.
 2. Visualize the matrix using the built-in plotting function for easier interpretation.
References
References
 • RetroCalc GitHub Repository: https://github.com/mdnx/RetroCalc
 • Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy.
Protocol references
Deeks JJ, Bossuyt PM. Chapter 3: Methods and presentation of systematic reviews of test accuracy. In: Deeks JJ, Bossuyt PM, Leeflang MM, Takwoingi Y editor(s). Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 2.0 (updated July 2023).