Integrated Test & Evaluation

Applying advanced statistical techniques to make DoD test & evaluation more efficient

Test and evaluation (T&E) is a critical component of the Department of Defense’s (DoD’s) process for acquiring new systems because it provides essential assessment of the system’s capabilities or deficiencies to help guide decision-making. However, the DoD faces unique challenges in designing and implementing quality tests due to the complexity of the systems and the difficulty and cost associated with approximating operational environments. While testing occurs as a system is being developed, estimation of operational behavior has traditionally been based on testing that occurs once a system has reached a near-final state.

The goal of this work has been to develop and apply advanced statistical techniques – Bayesian inference and design of experiments in particular – for integrating information from multiple phases of test, helping the DoD to more rapidly acquire and field new and effective capabilities.

Left: A comparison of Bayesian posterior estimates for various approaches to integrated test information. Middle: Lower credible bounds from Bayesian posteriors for binary test results, by number of tests and number of failures. Right: A summary of Bayesian prior predictive distribution for binary test results.

Relevant Publications:

  • Krometis, Justin, Kyle Provost, Corinne Stafford, Victoria Sieck, and Laura Freeman. “A Comparison of Bayesian Methods for Integrating Information from Developmental and Operational Test and Evaluation.” Military Operations Research Journal (accepted, in press).
  • Sieck, Victoria R.C., Justin A. Krometis, and Steven Thorsen. “A Framework for Using Priors in a Continuum of Testing.” Military Operations Research 29, no. 3 (2024). https://doi.org/10.5711/1082598329319.
  • Risher, Kyle, Daniel Wolodkin, Justin Krometis, Victoria Sieck, Laura Freeman, Jeremy Werner, and Sandra Hobson. “Maximizing the Use of Data to Make Winning Decisions in the Face of Finite Resources.” Naval Engineers Journal 136, no. 1–2 (Spring 2024): 111–20.