Multi-informant approaches to mental health assessment
By Andres De Los Reyes
Andres De Los Reyes received his PhD from Yale University (2008) and is an assistant professor in the Department of Psychology at the University of Maryland, College Park. His research focuses on understanding contextual variations in mental health concerns, with an emphasis on social anxiety. The goal of this work involves developing new methods for personalizing mental health care to improve patient outcomes. He received the 2013 Distinguished Scientific Award for Early Career Contribution to Psychology (Applied Research) from the American Psychological Association, as well as the 2013 Early Career Research Contribution Award from the Society for Research in Child Development. He serves as associate editor for the Journal of Clinical Child and Adolescent Psychology, Journal of Child and Family Studies and Journal of Psychopathology and Behavioral Assessment. He is on the editorial boards of such journals as the Journal of Consulting and Clinical Psychology, Psychological Assessment and Behavior Therapy. Author website.
Patients experiencing mental health concerns lead complex lives. Indeed, mental health concerns likely arise out of an intricate interplay among biological, psychological and socio-cultural factors that pose risk for, or offer protection against, the expression of maladaptive reactions to environmental circumstances or social contexts (e.g., Cicchetti, 1984; Luthar, Cicchetti, & Becker, 2000; Sanislow et al., 2010). However, contexts may vary in their propensity to elicit mental health concerns (e.g., Mischel & Shoda, 1995). Consequently, patients exhibit the potential to experience mental health concerns within some contexts, such as the home setting or within peer interactions, but not others, such as work or school settings. In fact, a great deal of evidence indicates that contextual variations in mental health concerns occur within a variety of mental health domains including social anxiety, attention and hyperactivity, aggressive and antisocial behavior and substance use (e.g., Bögels et al., 2010; Dirks, De Los Reyes, Briggs-Gowan, Cella, & Wakschlag, 2012; Kraemer et al., 2003). As a result, clinicians might "miss" identifying mental health concerns if their assessments do not adequately account for contextual variations in mental health. Thus, when assessing a patient, a clinician often has to take a comprehensive approach that involves collecting reports from multiple informants. These informants may include significant others in the patient’s life (e.g., spouses in the case of adults, and parents and teachers in the case of children and adolescents), as well as the patient herself or himself (De Los Reyes, Thomas, Goodman, & Kundey, 2013). Trained raters of mental health concerns, such as clinical interviewers, might also contribute reports within these assessments.
Clinicians may use multiple informants’ reports to make decisions about mental health care, such as assigning diagnoses and planning treatment (e.g., Hawley & Weisz, 2003). Researchers may use these multiple reports to draw conclusions from the findings of empirical work, such as research seeking to identify treatments that successfully ameliorate mental health concerns (i.e., evidence-based treatments; Weisz, Jensen Doss, & Hawley, 2005). In both practice and research settings, collecting multiple informants’ reports generates a great deal of information about patients’ mental health concerns. However, the individual reports often yield inconsistent conclusions (i.e., “informant discrepancies”; Achenbach, 2006; De Los Reyes & Kazdin, 2005, 2006). For instance, an adolescent patient in a clinical intake assessment may be identified as experiencing “low” mood based on a parent or teacher report about the adolescent; whereas the adolescent may self-report that she experiences her mood as “elevated.” Similarly, a study of a treatment for adult social anxiety may reveal that the treatment successfully reduces social anxiety symptoms when based on clinicians’ reports, whereas patients’ self-reports may indicate that no such reductions occurred.
Historically, informant discrepancies have created considerable uncertainty as to how best to care for patients and draw conclusions from research (De Los Reyes, 2013). Perhaps a reason for these uncertainties is that much of the research on informant discrepancies has focused on the extent to which these discrepancies reflect measurement error or informants’ reporting biases (e.g., De Los Reyes, 2011; De Los Reyes, Kundey, & Wang, 2011; Richters, 1992). In contrast, a great deal of theoretical and methodological work, as well as current thinking on best practices in mental health assessment, assume that the value in multi-informant assessments lies in capturing the unique perspectives held by each informant providing a report (e.g., Achenbach, McConaughy, & Howell, 1987; Hunsley & Mash, 2007). For example, discrepancies among multiple informants’ reports may reflect meaningful contextual variation in patients’ mental health concerns, particularly if informants vary in the contexts within which they observe patients (e.g., home vs. school or work settings; Kraemer et al., 2003).
Importantly, recent research has weighed in on these different ideas about informant discrepancies and what they represent. In fact, this research may reduce the uncertainties that currently exist when using and interpreting multiple informants’ reports. Further, much of this research has focused on mental health assessments of children and adolescents (De Los Reyes, Thomas et al., 2013). Yet, mental health researchers are beginning to make inroads in understanding how informant discrepancies operate within adult mental health assessments (Achenbach, Krukowski, Dumenci, & Ivanova, 2005; De Los Reyes, Bunnell, & Beidel, 2013; Oltmanns & Turkheimer, 2009; van der Ende, Verhulst, & Tiemeier, 2012). In this Science Brief, I review this work and how it may inform mental health practice and research.
What might discrepancies within multi-informant assessments reflect?
Much of the recent research on multi-informant assessment flows from the idea that commonly used informants within mental health assessments often vary in where they observe patients. For instance, mental health assessments for child and adolescent patients might rely on reports taken from parents and teachers, who typically observe children and adolescents within home and school contexts, respectively (Achenbach et al., 1987; Kraemer et al., 2003). Assessments of adult patients might follow a similar approach in that these assessments often consist of both patients’ self-reports and clinicians’ reports about patients (Achenbach et al., 2005). These reports might also contextually vary. Specifically, patients may base their reports on how they behave in work and home settings, whereas clinicians may take this information into account but also focus on how the patient behaves in the clinic setting (De Los Reyes, Bunnell et al., 2013). In both of these cases, informants differ in their opportunities for observing mental health concerns. Thus, if patients do indeed contextually vary in where they express concerns, then discrepancies among informants’ reports should, in part, reflect these contextual variations.
Recent work supports these notions about informant discrepancies. For example, a study of 327 preschool children aged 3-5 years found that children varied in the contexts within which they expressed disruptive behavior symptoms (De Los Reyes, Henry, Tolan, & Wakschlag, 2009). That is, some children expressed disruptive behavior exclusively within interactions with parental adults, some exclusively within interactions with non-parental adults (i.e., unfamiliar clinical examiners), other children across interactions with parental and non-parental adults and, still other children, did not express disruptive behavior within any of these interactions. In this same study, parents’ reports of children’s disruptive behavior more closely “matched” how children behaved within interactions with parental adults than with non-parental adults, whereas teachers’ reports more closely matched how children behaved within interactions with non-parental adults than with parental adults. Importantly, similar effects have been observed in a study of relatively older children (N=123 children; mean age=13.30), which focused on links between parents’ and teachers’ reports of aggressive behavior and the social experiences that tended to elicit these behaviors (Hartley, Zakriski, & Wright, 2011). Specifically, Hartley et al. (2011) found that as parents and teachers provided increasingly similar reports of children’s aggressive behavior, so did the similarities increase between the kinds of social experiences within which parents and teachers reported observing the children exhibiting aggressive behavior (e.g., peer interactions or receiving instructions from adult authority figures). Overall, in assessments of children expressing observable mental health concerns (i.e., aggressive and disruptive behavior), informants who systematically differ in where they observe children (i.e., parents and teachers) provide reports that appear to map onto contextual variations in children’s mental health concerns.
Links between multi-informant assessments and contextual variations in mental health can also be found in assessments of adult patients. For example, to make sense of discrepancies between patients’ self-reports and clinicians’ reports in social anxiety assessments, researchers recently administered a series of social tasks (e.g., one-on-one structured and unstructured interactions; impromptu speech) to adult social anxiety patients (De Los Reyes, Bunnell et al., 2013). This contextually sensitive assessment allowed for the classification of patients’ social competence deficits (i.e., problems with expressing socially appropriate or normative behavior within social interactions) into two profiles: (1) deficits across tasks and (2) deficits within specific tasks. Importantly, this study linked informants’ symptom reports to these tasks. Specifically, patients’ and clinicians’ reports of the patients’ symptoms were more likely to agree than disagree when patients exhibited social competence deficits consistently across social tasks. Similar to assessments of children’s aggressive and disruptive behavior (De Los Reyes et al., 2009; Hartley et al., 2011), these findings provide important support for the idea that discrepancies between patients’ and clinicians’ mental health reports are ecologically valid reflections of contextual variations in patients’ mental health concerns.
Can multi-informant assessment research inform patient care?
A key implication of the work reviewed above is that patients may vary in their clinical presentation. Patients may express mental health concerns within some contexts and not others. This possibility of within-patient contextual variations in mental health concerns indicates that a single patient may express concerns within, for instance, work settings but not home settings. There also exists the possibility of significant between-patient contextual variations. That is, multiple patients may vary in terms of the specific contexts within which they express mental health concerns. For example, Patient A may experience mental health concerns at work but not home, whereas Patient B may experience concerns at home but not work.
The understanding that contextual variations in mental health concerns may occur within any one patient, and also between any two patients, points to a key idea. Specifically, not all patients require or may be suited for the exact same mental health assessments or interventions administered in a single way (e.g., National Institute of Mental Health [NIMH], 2008). In line with this view, providing proper patient care may involve clinicians seeking to personalize or tailor assessments and interventions to patients’ unique needs. Similarly, researchers may develop assessments and interventions that can be systematically modified to target populations that exhibit significant within- and between-patient contextual variations in mental health concerns. To this end, multi-informant assessments may assist in both detecting contextual variations in mental health concerns and informing the development of personalized assessment and intervention techniques.
The NIMH recently released a strategic plan regarding its current funding priorities, which include broadening outcome measurement so as to personalize assessments and interventions to patients’ unique needs (Strategy 3.2; NIMH, 2008). Consequently, if a clinician wishes to administer a mental health assessment that is personalized to the unique needs of each patient, the work reviewed above indicates that this assessment should be able to capture information about the specific contexts within which the patient requires care. In this respect, future work in multi-informant assessment may inform the development of contextually sensitive assessments that improve patient care. That is, developing contextually sensitive mental health assessments may allow clinicians to identify treatment techniques that best “fit” what the patient needs in order to achieve significant symptom reduction and improved functioning. These issues merit further study.
I have summarized recent research investigating the value of the discrepancies commonly observed within multi-informant mental health assessments. Findings from recent studies of both child and adult patients reveal circumstances in which informant discrepancies yield meaningful information about contextual variations in patients’ mental health concerns. As a result, future research on multi-informant assessments may inform the development of contextually sensitive assessment paradigms that clinicians can use to personalize assessment and intervention techniques to fit patients’ unique needs. Importantly, the research discussed here largely focused on assessments of observable mental health concerns in children (e.g., aggressive and disruptive behavior), with some limited research focused on adult patients (e.g., Achenbach et al., 2005; De Los Reyes, Bunnell et al., 2013; De Los Reyes, Thomas et al., 2013; Oltmanns & Turkheimer, 2009; van der Ende et al., 2012). These issues merit further study in patients exhibiting varied mental health concerns (e.g., attention problems and hyperactivity, depressive symptoms and substance use) and across a wider range of developmental periods (e.g., childhood, adolescence, emerging adulthood and older adulthood). In sum, identifying when informant discrepancies reveal information about the specific contexts within which patients express mental health concerns may improve how clinicians and researchers use this information when providing patient care. Advancements of this sort may lead to improved methods for personalizing or tailoring mental health assessments and interventions to address patients’ unique needs.
This research program has been supported by the National Institutes of Health (MH67540; DA033913; DA018647-05S1) and the College of Behavioral and Social Sciences at the University of Maryland at College Park. I also have benefited greatly from a network of colleagues who helped me build this research program, including Alan Kazdin, Mitch Prinstein, Lauren Wakschlag, Patrick Tolan, David Henry, Deborah Beidel, Wendy Kliewer, Carl Lejuez, Laura MacPherson, Candice Alfano, Mo Wang, Kimberly Goodman, Eric Youngstrom, Amelia Aldao, Jessecae Marsh, Matthew Lerner and my graduate students: Sarah Thomas, Tara Augenstein and Melanie Lipton.
Achenbach, T.M. (2006). As others see us: Clinical and research implications of cross-informant correlations for psychopathology. Current Directions in Psychological Science, 15, 94-98. doi: 10.1111/j.0963-7214.2006.00414.x
Achenbach, T.M., Krukowski, R.A., Dumenci, L., & Ivanova, M.Y. (2005). Assessment of adult psychopathology: Meta-analyses and implications of cross-informant correlations. Psychological Bulletin, 131, 361-382. doi: 10.1037/0033-2909.131.3.361
Achenbach, T.M., McConaughy, S.H., & Howell, C.T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213-232. doi: 10.1037/0033-2909.101.2.213
Bögels, S.M., Alden, L., Beidel, D.C., Clark, L.A., Pine, D.S., Stein, M.B., & Voncken, M. (2010). Social anxiety disorder: Questions and answers for the DSM-V. Depression and Anxiety, 27, 168-189. doi: 10.1002/da.20670
Cicchetti, D. (1984). The emergence of developmental psychopathology. Child Development, 55, 1-7. doi: 10.2307/1129830
De Los Reyes, A. (2011). Introduction to the special section. More than measurement error: Discovering meaning behind informant discrepancies in clinical assessments of children and adolescents. Journal of Clinical Child and Adolescent Psychology, 40, 1-9. doi: 10.1080/15374416.2011.533405
De Los Reyes, A. (2013). Strategic objectives for improving understanding of informant discrepancies in developmental psychopathology research. Development and Psychopathology, 25, 669-682. doi: 10.1017/S0954579413000096
De Los Reyes, A., Bunnell, B.E., & Beidel, D.C. (2013). Informant discrepancies in adult social anxiety disorder assessments: Links with contextual variations in observed behavior. Journal of Abnormal Psychology, 122, 376-386. doi: 10.1037/a0031150
De Los Reyes, A., Henry, D.B., Tolan, P.H., & Wakschlag, L.S. (2009). Linking informant discrepancies to observed variations in young children’s disruptive behavior. Journal of Abnormal Child Psychology, 37, 637-652. doi: 10.1007/s10802-009-9307-3
De Los Reyes, A., & Kazdin, A.E. (2005). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131, 483-509. doi: 10.1037/0033-2909.131.4.483
De Los Reyes, A., & Kazdin, A.E. (2006). Conceptualizing changes in behavior in intervention research: The range of possible changes model. Psychological Review, 113, 554-583. doi: 10.1037/0033-295X.113.3.554
De Los Reyes, A., Kundey, S.M.A., & Wang, M. (2011). The end of the primary outcome measure: A research agenda for constructing its replacement. Clinical Psychology Review, 31, 829-838. doi: 10.1016/j.cpr.2011.03.011
De Los Reyes, A., Thomas, S.A., Goodman, K.L., & Kundey, S.M.A. (2013). Principles underlying the use of multiple informants’ reports. Annual Review of Clinical Psychology, 9, 123-149. doi: 10.1146/annurev-clinpsy-050212-185617
Dirks, M.A., De Los Reyes, A., Briggs-Gowan, M.J., Cella, D., & Wakschlag, L.S. (2012). Embracing not erasing contextual variability in children’s behavior: Theory and utility in the selection and use of measures and informants in the assessment of developmental psychopathology. Journal of Child Psychology and Psychiatry, 53, 558-574. doi: 10.1111/j.1469-7610.2012.02537.x
Hartley, A.G., Zakriski, A.L., & Wright, J.C. (2011). Probing the depths of informant discrepancies: Contextual influences on divergence and convergence. Journal of Clinical Child and Adolescent Psychology, 40, 54-66. doi: 10.1080/15374416.2011.533404
Hawley, K.M., & Weisz, J.R. (2003). Child, parent, and therapist (dis)agreement on target problems in outpatient therapy: The therapist’s dilemma and its implications. Journal of Consulting and Clinical Psychology, 71, 62-70. doi: 10.1037/0022-006X.71.1.62
Hunsley, J., & Mash, E.J. (2007). Evidence-based assessment. Annual Review of Clinical Psychology, 3, 29-51. doi: 10.1146/annurev.clinpsy.3.022806.091419
Kraemer, H.C., Measelle, J.R., Ablow, J.C., Essex, M.J., Boyce, W.T., & Kupfer, D.J. (2003). A new approach to integrating data from multiple informants in psychiatric assessment and research: Mixing and matching contexts and perspectives. American Journal of Psychiatry, 160, 1566-1577. doi: 10.1176/appi.ajp.160.9.1566
Luthar, S.S., Cicchetti, D., & Becker, B. (2000). The construct of resilience: A critical evaluation and guidelines for future work. Child Development, 71, 543-562. doi: 10.1111/1467-8624.00164
Mischel, W., & Shoda, Y. (1995). A cognitive-affective system theory of personality: Reconceptualizing situations, dispositions, dynamics, and invariance in personality structure. Psychological Review, 102, 246-268. doi: 10.1037/0033-295X.102.2.246
National Institute of Mental Health. (2008). The National Institute of Mental Health strategic plan (NIH Publication No. 08-6368). Retrieved May 31, 2013
Oltmanns, T.F., & Turkheimer, E. (2009). Person perception and personality pathology. Current Directions in Psychological Science, 18, 32-36. doi: 10.1111/j.1467-8721.2009.01601.x
Richters, J.E. (1992). Depressed mothers as informants about their children: A critical review of the evidence for distortion. Psychological Bulletin, 112, 485-499. doi: 10.1037/0033 2909.112.3.485
Sanislow, C.A., Pine, D.S., Quinn, K.J., Kozak, M.J., Garvey, M.A., Heinssen, R.K., Wang, P.S., & Cuthbert, B.N. (2010). Developing constructs for psychopathology research: Research domain criteria. Journal of Abnormal Psychology, 119, 631-639. doi: 10.1037/a0020909
van der Ende, J., Verhulst, F.C., & Tiemeier, H. (2012). Agreement of informants on emotional and behavioral problems from childhood to adulthood. Psychological Assessment, 24, 293-300. doi: 10.1037/a0025500
Weisz, J.R., Jensen Doss, A., & Hawley, K.M. (2005). Youth psychotherapy outcome research: A review and critique of the evidence base. Annual Review of Psychology, 56, 337-363. doi: 10.1146/annurev.psych.55.090902.141449
The views expressed in Science Briefs are those of the authors and do not reflect the opinions or policies of APA.