Criteria for Evaluating Treatment Guidelines


This document presents a set of criteria to be used in evaluating treatment guidelines that have been promulgated by health care organizations, government agencies, professional associations, or other entities.1 Although originally developed for mental health interventions, the criteria presented are equally applicable in other health service areas.

Two factors prompted this effort by the American Psychological Association (APA) to create a policy basis for evaluating guidelines. First, guidelines of varying quality, from both public and private sources, have been proliferating. Second, the interest and expertise in methodological issues within the profession of psychology made it likely that APA could make a useful contribution to the evaluation of treatment guidelines.

Generally, health care guidelines are pronouncements, statements, or declarations that suggest or recommend specific professional behavior, endeavor, or conduct in the delivery of health care services. Guidelines are promulgated to encourage high quality care. Ideally, they are not promulgated as a means of establishing the identity of a particular professional group or specialty, nor are they used to exclude certain persons from practicing in a particular area.

There are two different types of health care guidelines: practice guidelines and treatment guidelines. Practice guidelines, which are not addressed in this document, consist of recommendations to professionals concerning their conduct and the issues to be considered in particular areas of clinical practice rather than on patient outcomes or recommendations for specific treatments or specific clinical procedures at the patient level. Treatment guidelines, which are the focus of this document, provide specific recommendations about treatments to be offered to patients. That is, treatment guidelines are patient directed or patient focused as opposed to practitioner focused, and they tend to be condition or treatment specific (e.g., pediatric immunizations, mammography, depression).

The purpose of treatment guidelines is to educate health care professionals2 and health care systems about the most effective treatments available. When there is sufficient information and the guidelines are done well, they can be a powerful way to help translate the current body of knowledge into actual clinical practice.

Many treatment guidelines are disorder based. The most common classification system is the International Classification of Diseases (ICD-10; World Health Organization, 1992) and, for mental disorders, the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Association, 1994). The disorder-based approach has limitations: Patients3 commonly present issues that cut across diagnostic lines, dual diagnoses (comorbidity) are common, and disorder-based diagnosis is often a weak basis for determining appropriate levels of care and other characteristics of treatment. Other classification systems, such as the World Health Organization's functionally based International Classification of Functioning, Disability, and Health (World Health Organization, 2001), might also provide a basis for the development of treatment guidelines. It is important for groups constructing or evaluating guidelines to consider the adequacy and limitations of the nosological systems on which they are based.

Health care professionals are in the best position to be aware of the unique characteristics of individual patients. The treatment strategy most likely to succeed usually combines the most effective specific interventions with a strong therapeutic relationship and a mutual expectation of and framework for improvement. Such factors, which are common to most treatment situations, can be powerful determinants of treatment success. Good guidelines allow for flexibility in treatment selection so as to maximize the range of choices among effective treatment alternatives. The judgment of health care professionals, although always needed, is particularly important in the treatment of conditions for which research data are limited. Guideline panels should take these factors into consideration and particularly should avoid encouraging an overly mechanistic approach that could undermine the treatment relationship.

It is often assumed that the use of treatment guidelines will significantly reduce the cost of services. This is not necessarily true. It is possible that guideline implementation may cause some services to be discontinued because of evidence documenting an intervention's lack of efficacy. However, it is also possible that the adoption of guidelines will lead to a shift toward more effective but not necessarily less costly services. And it is possible that more costly or additional treatments will be recommended.

Another common assumption is that standardizing treatment via guidelines will always be beneficial because it reduces practice variation. However, variation in clinical practice is often based on the needs of individual patients and their responses to specific treatments. When the application of guidelines results in a rigid system that eliminates the ability to respond to individual needs of the patient and the opportunity for self-correction in treatment, this can be detrimental to patient care.

In this document, it is not presumed that guidelines are inherently either beneficial or detrimental, and the document is not intended either to encourage or to discourage their development. However, the burden of proof remains on the makers of each guideline and those responsible for its implementation to establish that the application of the guideline is indeed beneficial and does not impair patient care.

The purpose of this document is to provide criteria to assist in the determination of the strengths and weaknesses of each guideline. These criteria are intended to provide structure and guidance for those individuals or groups that evaluate the quality and appropriateness of treatment guidelines. Each criterion describes an important issue that guideline makers should aspire to address in the best possible manner.

The primary purpose of this document is to assist in the evaluation of treatment guidelines. Although it will be helpful to those wishing to construct treatment guidelines, it does not provide sufficient specificity to serve as the sole basis for such efforts. It is not intended to promote the application of a particular set of treatment techniques or approaches. Finally, this document is not intended to imply that the treatments provided by individual practitioners should be subject to the evaluative process described here for assessing treatment guidelines.

Treatment guidelines have the potential to influence the health care of many patients, and therefore the guidelines and the process used in their development should be open to public scrutiny. Moreover, failure to disclose the scientific justification for a guideline violates a basic principle of science, which requires open scrutiny and debate. Without the disclosure of adequate scientific information, guidelines are mere expressions of opinion.

This document is organized on the basis of two related dimensions for the evaluation of guidelines. The first dimension is treatment efficacy, the systematic and scientific evaluation of whether a treatment works. The second dimension is clinical utility, the applicability, feasibility, and usefulness of the intervention in the local or specific setting where it is to be offered. This dimension also includes determination of the generalizability of an intervention whose efficacy has been established. To encourage accountability, criteria for evaluating the process of guideline production are also provided.

1It is important to recognize that the term guidelines generally refers to pronouncements that support or recommend but do not mandate specific approaches or actions. In this regard, guidelines differ from what are sometimes called standards in that standards are considered mandatory and may be accompanied by an enforcement mechanism. The Criteria for Evaluating Treatment Guidelines should be regarded as guidelines, which means that it is essentially aspirational in intent. It is intended to facilitate and assist the evaluation of treatment guidelines but is not intended to be mandatory, exhaustive, or definitive and may not be applicable to every situation.
2We have chosen to use the term health care professional, shortened at times to professional, to refer to the trained and legally authorized person who delivers health care services.
3To be consistent with the context in which most guidelines are applied, we use the term patient to refer to the individual (child or adult), couple, family, or group receiving treatment. However, we also recognize that in many situations, there are important and valid arguments for using terms such as client, consumer, or person in place of patient to describe the recipient of services.
  • Page:
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6