While screening can include a wide range of activities (e.g., laboratory tests, genetic screening), in the criminal justice system screening typically refers to a brief, but systematic process to aid in the detection of mental illness. Screening tools can often be administered by nonmental health professionals, including correctional officers. They may also be self-report questionnaires, which can be completed either by paper or on a computer. They are not intended to make a diagnosis, rather they aim to distinguish persons who likely have an illness from those who are unlikely to have an illness. Individuals who screen positive should be seen by a mental health professional for a full assessment to arrive at a diagnosis and treatment plan as appropriate.
People with mental illness are overrepresented in the criminal justice system. These inmates often have poorer outcomes including difficulties completing correctional programs, self-harm and suicide, institutional violence, and more time in segregation. Studies in various countries have reported that only 25% of inmates with mental illness receive treatment in jails and prisons. Screening is proposed as a solution to address this issue by supporting improved detection of mental illness. The end goal is to improve outcomes of offenders with mental illness. Several criteria have been proposed to evaluate how effective screening is at achieving this goal. The best known criteria are a series of 10 criteria proposed by James Maxwell Glover Wilson and Gunnar Jungner for the World Health Organization in 1968. This article reviews four criteria that influence the effectiveness of screening:
- the accuracy of the screening tool
- definitions of mental illness
- the follow-up assessment and treatment services provided
- the needs and treatability of those who are newly identified by screening.
Accuracy of Screening Tools
Whether a screening tool is accurate is typically determined by comparing the screening result to a gold standard assessment that is thought of as being the best measure of the person’s true mental health status. This might include a structured diagnostic interview (e.g., the Structural Clinical Interview for DSM Disorders) or an in-depth clinical assessment. If a person is deemed highly likely to be ill on the screening test (i.e., a positive screen) and diagnosed as ill on the gold standard measure, the person is classified as a true positive. Conversely, a person is classified as a true negative if both the screen and the gold standard measure agree that the person is not ill. False positives (i.e., a positive screen for a person who is not ill according to the gold standard) and false negatives (i.e., a negative screen for a person who is ill according to the gold standard) thus represent screening errors.
There are a number of statistics that can be used to estimate the accuracy of a screening tool. Sensitivity and specificity are the two most common statistics used for this purpose. The sensitivity represents the percentage of people who have an illness who are identified by the screening tool as having a probable mental illness. That is why it is also referred to as the true positive rate of a screening tool. The specificity, or the true negative rate of the test, is the percentage of people who do not have an illness who are identified by the screening tool as unlikely to have a mental illness. When developing or implementing a test, one must decide the relative importance of true and false positives. On most tests, a cutoff score is used to convert total scores into those who are classified as positive or negative screens. A lower cutoff score will increase sensitivity but decrease specificity and would be favored if missing cases of illness is the top priority, and the harms associated with false positives are low. A higher score can be used to increase specificity but decrease sensitivity. This would be favored if the harms associated with false positives are high (e.g., resource implications for follow-up services, high risk of side effects if treatment is provided to a person who is not ill).
Sensitivity and specificity are commonly reported because they are in most cases constant properties of a test that are not affected by prevalence (however, this property will not hold, if the screening accuracy of the test varies between groups of differing individual or clinical characteristics, such as disease severity; this is referred to as spectrum bias). To illustrate, Prison A is a maximum security prison, and Prison B is a minimum security prison. Because inmates with mental illness have higher rates of risk factors and behaviors, the prevalence of mental illness is 35% at Prison A and only 5% at Prison B. If all inmates are screened upon their transfer into the prisons, one would expect the same percentage of inmates with mental illness to be identified by screening in both cases. This is because how a person is rated or scored on a screening test should reflect the person’s unobservable mental health status. If a person is ill, there is a certain likelihood that the person will report those symptoms (i.e., the sensitivity of the test). Similarly, among those who are not ill, the specificity is the likelihood that they will not report symptoms.
While sensitivity and specificity are useful from a research perspective, they are less useful to a clinician reviewing screening results. This is because sensitivity and specificity start from the outcome and work backward to predict the screening result. Clinicians do not know the inmate’s illness status (if they did, the assessment would not be needed) and therefore need to judge how accurate the screening result is. The positive and negative predictive value (NPV) of a test provide this information. The positive predictive value (PPV) indicates the percentage of people who are identified by screening as likely to have an illness (i.e., screen positive) who are in fact ill. The NPV indicates the percentage of people who are identified as unlikely to have an illness (i.e., screen negative) who are not ill. The limitation of PPV and NPV is that they are related to prevalence. The same test will have a higher PPV and a lower NPV in population with a higher prevalence (i.e., Prison A in the illustration) compared to a population with a lower prevalence (i.e., Prison B). If the prevalence is higher in the population, it must also be higher within the two groups defined by the screening result. Table 1 shows the calculations of these four statistics for Prisons A and B.
As seen in Table 1, the sensitivity (80%) and specificity (60%) of the test are the same in both prisons. However, in Prison A, 52% of inmates referred by screening have an illness, whereas only 10% of inmates referred by screening in Prison B have an illness. In other words, in Prison B, inmates referred to clinicians for follow-up assessments are much more likely to not have an illness, whereas in Prison A, approximately half of the inmates referred following screening will have an illness. The situation in Prison A reflects the major challenge of trying to predict rare events. This issue has received considerable attention in screening for rare outcomes such as suicide and self-harm.
A 2013 systematic review found that the best studied mental health screening tools for use in jails and prisons have a sensitivity between 60% and 75% and specificity between 50% and 75%. At these levels of accuracy, screening would detect 2–3 times as many inmates with illness as compared to studies in settings without screening but with a relatively large number of inmates without illness also requiring assessments. Many screening tools are less accurate when used with women offenders. There was little research conducted with minority ethnic and cultural groups identified in the review. Gender- and culturally sensitive screening are important gaps in research and practice. Since accurate diagnosis is a precursor to treatment that can improve outcomes for inmates with mental illness, well-validated screening tools are an important component of a correctional mental health strategy. The following sections discuss some of the issues that must be attended to once an accurate screening tool is identified.
Diagnosis Definitions and Screening
There are a number of important issues surrounding the definition of mental illness that will affect decisions about when to screen and how effective screening is. These include the following: (a) the distinction between incident and prevalent illnesses, (b) whether there is a presymptomatic (or prodromal) phase during which early intervention can prevent onset or course or illness, and (c) overdiagnosis.
Incident Versus Prevalent Illness
The distinction between incidence and prevalence is critical to understand the risk that a person with mental illness poses. Incidence (or an incident illness in the individual case) refers to a new illness that has its onset within the days or weeks prior to screening. Prevalence (or a prevalent illness) refers to a longstanding illness, which, in the case of discussing newly detected cases through screening, was previously undiagnosed. This question is important in terms of determining the timing at which screening should be offered. Most research on screening in jails and prisons focuses on the intake period. In the context of suicide risk screening and assessment, this is often recommended following transfers between institutions. If most illness in prisons predates incarceration (i.e., is prevalent), then intake screening is an appropriate strategy. If screening seeks to identify illness that has its onset during incarceration, then repeated screening at critical risk periods or at a regular interval is needed.
Prodromal or Presymptomatic Phase
In many health screening contexts (e.g., cancer), the goal is to identify abnormalities before they cause the individual to experience symptoms. This is less common in mental health screening, although some researchers have explored the preventive value of identifying prodromal symptoms (i.e., early signs that indicate a developing mental illness, prior to its actual occurrence), especially in schizophrenia and other psychotic disorders. Given that the age distribution of jail and prison populations overlaps with the typical time of onset of many mental illnesses, some have argued for screening for prodromal symptoms. This area is much less developed than other areas of screening, possibly owing to challenges in defining and measuring prodromal symptoms.
Overdiagnosis
Overdiagnosis refers to the situation in which an individual may be diagnosed with an illness that would not have been diagnosed in the absence of screening, but the diagnosis is not associated with impaired functioning or increased risk of adverse outcomes. In other words, the prevalence rate of the illness may be higher after introducing screening, but the newly identified cases are mild in severity. This should not arise in the context of diagnoses of mental illness made by a mental health professional given that most diagnostic criteria include a functional impairment criterion. However, critics of diagnostic classification systems (e.g., the Diagnostic and Statistical Manual of Mental Disorders; International Classification of Diseases) argue that each revision has lowered the threshold for what constitutes an illness and thus increased overdiagnosis. Overdiagnosis has been raised in many other areas of medical screening (most notably cancer). Discussion of overdiagnosis in the context of mental health screening has only begun since about 2005 in the general population, and this issue has not been discussed in a criminal justice context where the emphasis has been largely on preventing missed cases of illness. If screening results in overdiagnosis, it may appear as though screening is ineffective as the prevalence of mental illness and potentially the rate of treatment use will increase. However, since the new service users are low risk, there may be no change in long-term outcomes. Overdiagnosis may also create a situation in which the highest needs cases do not receive the treatment needed due to a lack of resources to meet the demand created by screening. In this case, screening could lead to higher rates of adverse outcomes due to undertreatment.
Follow-Up Assessment and Treatment Services
As screening is not meant to determine an offender’s diagnosis by itself, follow-up services by a mental health professional are required. Ensuring that there are sufficient resources to provide timely structured assessments for all those who are referred by the screening tool and treatment for all those who are identified by screening as likely to be ill is essential. These follow-up services represent an opportunity to identify mistakes that are made by screening tests (i.e., to rule out illness for inmates without illness who are incorrectly referred). However, if the PPV of a test is low, there is a greater risk that an individual with mental illness may be overlooked by the clinician. Conversely, if clinicians place too much emphasis on screening results and set out to confirm them, there is a risk of starting treatment for an inmate without mental illness, which may have important side effects or contribute to stigma.
Needs and Treatability of Newly Identified Cases
In most medical contexts, screening is offered only to those who are not known to have an illness; in mental health research and practice, screening has often been offered to all individuals. Many mental health screening tools (e.g., the Brief Jail Mental Health Screen, Jail Screening Assessment Tool) include items about prior mental health service use, owing to fragmented service delivery among civil, forensic, and correctional mental health services. Despite their illness being known to a service provider, this information is often not available quickly enough to staff in jails and prisons to act on it. Most studies that have looked at detection of mental illness in prison have reported that detection is reasonably high for inmates who used mental health services in the community, and thus screening may have little or no impact on the outcomes of these inmates since they would likely be identified and treated even in the absence of screening. However, this finding is not consistent across countries, and given the importance of ensuring continuity of care, effective strategies for obtaining inmates’ mental health histories (either through improved information sharing or self-report screening) are essential. As of the end of 2015, no research looking at the needs and treatability of inmates whose mental illness is first detected by screening had been conducted. Studies in primary care in the community have found no benefit of screening in terms of symptom reduction associated with screening, and that psychotherapy is less effective for individuals who are referred by screening compared to those who are referred by a physician or other health professional. Given that rates of detection of illness are lower in correctional settings than in the community, and inmates have many cooccurring health and social functioning needs, it is unclear whether these findings generalize to the jail and prison context. The question of long-term outcomes following screening represents an important clinical and research question moving forward, that has only recently begun to attract attention.
References:
- Crumlish, N., & Kelly, B. D. (2009). How psychiatrists think. Advances in Psychiatric Treatment, 15(1), 72–79. doi:10.1192/apt.bp.107.005298
- Cuijpers, P., van Straten, A., van Schaik, A., & Andersson, G. (2009). Psychological treatment of depression in primary care: A meta-analysis. British Journal of General Practice, 59(559), 51–60. doi:10.3399/bjgp09X395139
- Gilbody, S., Sheldon, T., & House, A. (2008). Screening and case-finding instruments for depression: A metaanalysis. Canadian Medical Association Journal, 178(6), 1–11. doi:10.1503/cmaj.070281
- Glaros, A. G., & Kline, R. B. (1988). Understanding the accuracy of tests with cutting scores: The sensitivity, specificity, and predictive value model. Journal of Clinical Psychology, 44(6), 1013–1023.
- Martin, M. S., Colman, I., Simpson, A. I., & McKenzie, K. (2013). Mental health screening tools in correctional institutions: A systematic review. BMC Psychiatry, 13, 275. doi:10.1186/1471-244X-13-275
- Stein, D. J., Phillips, K. A., Bolton, D., Fulford, K., Sadler, J. Z., & Kendler, K. S. (2010). What is a mental/ psychiatric disorder? From DSM-IV to DSM-V. Psychological Medicine, 40(11), 1759–1765. doi:10.1017/S0033291709992261
- Wilson, J. M., & Jungner, Y. G. (1968). Principles and practice of mass screening for disease (No. 34). Geneva, Switzerland: World Health Organization.