F. Risk Prediction Errors
Risk prediction instruments, even those that are actuarial, are not perfect. One factor influencing the capacity to accurately predict behavior is called the base rate, the frequency of the outcome of interest (e.g., arrest, conviction, incarceration) in the sample being used to create the instrument. The more infrequent a behavior, the greater the difficulty in successfully predicting it. Thus, researchers are more successful predicting general recidivism than specific recidivism, such as particular violent acts.
All prediction tools produce both correct and incorrect predictions, also referred to as prediction errors. There are two possible kinds of errors: false positives and false negatives. False positives refer to incorrect predictions that offenders will commit new crimes. False positive errors are committed when individuals who would not have committed new crimes are subjected to secure settings or restrictive community supervision. False negatives refer to incorrect predictions that offenders will not commit new crimes. False negative errors occur when incarceration or restrictive conditions of supervision are not imposed on offenders who will commit new crimes. The terms true and false refer to the accuracy of the prediction. The terms negative and positive refer to the content of the prediction, that is, exhibiting a behavior (positive) or not exhibiting it (negative).
Because prediction errors with the current state of knowledge cannot be eliminated, the best to hope for is that their frequency can be reduced.With respect to any particular instrument, however, efforts to reduce one error type result in an increase in the other. To illustrate, suppose it is known from prior studies that 20% of felony offenders on community supervision can be expected to commit new felonies over the next 3 years. If there was a prediction instrument that was 100% accurate, identifying all of these recidivists-to-be would be very easy—the offenders could be rank-order classified by risk score and a cutoff established delineating the highest-scoring 20%. But when risk instruments are imperfect, setting the cutoff at the 20th percentile of scores means that some of the offenders designated as high risk will be false positives, and that some of the offenders whose scores fell outside the 20th percentile will be false negatives. By setting a more inclusive cutoff—for example, at the 30th percentile—the instrument will capture more true positives while at the same time increasing false positives.
What kinds of incorrect predictions most likely to occur depend upon the values attached to each kind of error. If those conducting the assessment are risk averse, meaning that they emphasize public safety and strive to avoid new victimizations, they are likely to set low cutoffs delineating high- from low-risk offenders, resulting in a greater number of offenders (justly or unjustly) falling into the category judged high risk. If the assessors value justice for the individual being sanctioned in an effort to avoid unnecessarily restrictive supervision or confinement, they would establish more stringent cutoffs, resulting in fewer offenders judged threats to the community. Which type of error is worse? That depends upon policymakers’ value frameworks.
Even though the best prediction instruments are subject to error, what is important to keep in mind is that good instruments yield predictions that improve upon chance or an officer’s subjective judgment. Using a valid and reliable risk assessment tool helps to avoid the higher rate of error associated with using no actuarial instrument at all.