The Influence of Experience and Input Information Fidelity Upon Posterior Probability Estimation in a Simulated Threat-Diagnosis System
Report Number: AMRL TR 65-25
Author(s): Schum, David A., Goldstein, Irwin L., Southard, Jack F.
Corporate Author(s): Ohio State University
Laboratory: Behavioral Sciences Laboratory
Date of Publication: 1965-04
Pages: 80
Contract: AF 33(657)-10763
DoD Project: 7184
DoD Task: 718403
Identifier: AD0615758
Abstract:
Two experiments are described in which posterior probability estimates made by humans are compared with similar estimates made by a computer using a modification of Bayes' theorem incorporating human estimates of P(D/H). The task was to estimate, on the basis of intelligence data from a simulated threat-evaluation situation, the likelihood of various alternative hypotheses that could account for the observed data. The purpose of the first experiment was to determine the effect of increased experience upon the human's ability to estimate posterior probabilities. The purpose of the second experiment was to compare human and automated posterior probability estimates under several levels of input data fidelity. It was predicted that, under low fidelity conditions, human posterior probability estimates would become increasingly inferior to automated solutions. This hypothesis was only partially confirmed. In both experiments, but particularly in the second, the humans provided higher posterior probability estimates than the certainty in the data justified. With respect to the desing of diagnostic systems, the present research tends to confirm the feasibility of automated Bayesian hypothesis-selection incorporating expert human estimates of the conditional probabilities P(D/H).
Provenance: RAF Centre of Aviation Medicine
Author(s): Schum, David A., Goldstein, Irwin L., Southard, Jack F.
Corporate Author(s): Ohio State University
Laboratory: Behavioral Sciences Laboratory
Date of Publication: 1965-04
Pages: 80
Contract: AF 33(657)-10763
DoD Project: 7184
DoD Task: 718403
Identifier: AD0615758
Abstract:
Two experiments are described in which posterior probability estimates made by humans are compared with similar estimates made by a computer using a modification of Bayes' theorem incorporating human estimates of P(D/H). The task was to estimate, on the basis of intelligence data from a simulated threat-evaluation situation, the likelihood of various alternative hypotheses that could account for the observed data. The purpose of the first experiment was to determine the effect of increased experience upon the human's ability to estimate posterior probabilities. The purpose of the second experiment was to compare human and automated posterior probability estimates under several levels of input data fidelity. It was predicted that, under low fidelity conditions, human posterior probability estimates would become increasingly inferior to automated solutions. This hypothesis was only partially confirmed. In both experiments, but particularly in the second, the humans provided higher posterior probability estimates than the certainty in the data justified. With respect to the desing of diagnostic systems, the present research tends to confirm the feasibility of automated Bayesian hypothesis-selection incorporating expert human estimates of the conditional probabilities P(D/H).
Provenance: RAF Centre of Aviation Medicine