2012 Education Leadership Conference: Promoting Quality
By Rebecca A. Clay
The theme of the 11th Annual Education Leadership Conference (ELC) — Promoting Quality — is central to APA's Education Directorate, said Cynthia D. Belar, PhD, the directorate's executive director.
“We need to advance the culture of continuous quality improvement in education and in our students,” said Belar. “That's a core principle for the profession of teaching — not just to ensure quality but the continual enhancement of quality; it's not a static process.”
This year's conference brought together more than 125 representatives from psychology education and training organizations, psychological membership groups, APA divisions and APA governance groups. The event took place September 8-11 in Washington, D.C.
Like previous ELCs, the conference had a triple mission: addressing issues of mutual concern to psychology education and training groups at all levels; promoting a shared identity among psychology's educators; and influencing public policy regarding education in psychology and psychology in education.
“You were selected by your organization to represent that organization because you're leaders in the field of education,” said Michael C. Roberts, PhD, chair of APA's Board of Educational Affairs (BEA). “We need your expertise for the discussion to be meaningful, and you also have a critical role in taking back to your organization what you learn as a result of your ELC participation.”
Belar kicked off the conference with a look at APA's own efforts to promote quality.
One key mechanism is awards that draw attention to high-quality efforts. BEA's Golden Psi Award, for example, is a brand-new program that recognizes schools that apply psychological and educational research to create environments that help students learn. BEA's Awards to Advance Interdisciplinary Education and Training in Psychology acknowledge education and training programs that promote interdisciplinary, multidisciplinary or interprofessional teaching, research or practice in psychology. In addition, APA recognizes high school researchers by participating in the Intel International Science and Engineering Fair.
APA also provides resources to promote quality education. For teachers at the precollege level, APA offers modules for teachers on classroom management and related topics; lesson plans; and an online laboratory of experiments that give students the chance to do hands-on data analysis. At the college and university level, resources include a new “Practical Guidebook for the Competency Benchmarks,” a “Competency Assessment Toolkit for Professional Psychology” and “The Assessment CyberGuide for Learning Goals and Outcomes,” which helps departments develop assessment plans.
“APA has also put money where its mouth is,” said Belar. In August, she pointed out, APA's Council of Representatives approved spending up to $3 million to help unaccredited internships earn APA accreditation. That decision, she explained, grew out of “growing concern about a two-tiered system of training.” Plus, she said, psychology will not be considered a serious player in the health care system unless its students meet the profession's own standards for quality. “Accreditation is as important to professional education and training as the peer review process is to scientific publication,” she said.
APA's greatest impact on quality has been through its development of guidelines and standards, said Belar. Examples include the “National Standards for High School Psychology Curricula,” “Guidelines for Preparing High School Psychology Teachers,” “Principles for Quality Undergraduate Education in Psychology” and “Guidelines for the Undergraduate Psychology Major.” APA's accreditation of doctoral, internship and postdoctoral programs and its system of approving continuing education sponsors represent two other key quality assurance efforts.
In addition, APA convenes collaborative efforts focused on quality. These include the Health Service Psychology Education Collaborative, the Institute of Medicine Global Forum on Innovation in Health Professional Education, and the White House Joining Forces Initiative.
“Our current effort at promoting quality is the conference that begins now,” said Belar. “We're counting on your active participation to ensure success.”
It's what leaders do
The conference's first full day began with a look at the increasing push for quality in higher education. Judith S. Eaton, PhD, president of the Council for Higher Education Accreditation (CHEA) and board chair of the New Leadership Alliance for Student Learning and Accountability, began by laying out several myths about quality. These include the idea that quality is just out there in the ether, that no one knows how to evaluate it, and that even if you could, doing so would be harmful. Others believe that you do not have to worry about quality anyway, because everyone knows it when they see it. “I challenge all those notions,” said Eaton. Instead, she said, educators must “assert quality.” Once those expectations are created, their next task is to examine whether those expectations are being met. If those in higher education do not take on these responsibilities, she warned, others will.
That is already starting to happen, said Eaton, explaining that the country is engaged in a significant debate about educational quality. “The public wants to know more than ever about what we do in higher education, how and what we teach, what our performance is, and what happens to students,” said Eaton. Part of that debate focuses on what counts as quality: student outcomes, evidence of intellectual development, or job placements. Who decides — faculty, students, administrators — is also under debate. The stakes are especially high given the amount of money involved, said Eaton. The federal government spends $175 billion a year in student aid, and the average loan debt of seniors graduating in 2010 was $25,000. The growing diversity of higher education, such as e-courses and education via social media, adds to the debate's complexity.
Existing tools can help address these challenges. Accreditation, for example, seeks not just to assure a threshold of quality but to improve it. Institutions' own program reviews and such efforts as College Portrait of Undergraduate Education and Liberal Education and America's Promise Alliance can also help. But other tools are not under higher education's control. U.S. News and World Report's rankings have become the “de facto quality standards,” said Eaton. The states and the federal government are increasingly involved. And new players, including the Gates Foundation, are funding tools for examining educational quality.
Eaton urged educators to promote quality by asking themselves the following questions: What are the goals of your program or institution, and are they focused on student learning? What counts as quality when it comes to meeting those goals? Are the goals benchmarked so they can be compared to similar work by other programs or institutions? What counts as evidence that the goals are being met? How can you inform the public about how well your students are achieving? And how can you improve quality? “We use large amounts of public and private money,” said Eaton. “We must be accountable.”
Federal regulatory efforts in teacher education and beyond
Next, Ronald W. Marx, PhD, dean of the College of Education and professor of educational psychology at the University of Arizona, shared what he called a “cautionary tale” about who has the authority to declare quality. “I'm going to tell a tale about an inside-the-Beltway fight from an outside-the-Beltway person,” said Marx, describing his participation in a U.S. Department of Education committee. “I thought it was vitally important that a psychologist, especially a psychologist like me who has spent his career as a researcher, be part of the debate about teacher quality.”
Marx began by describing growing concern about teacher quality among the public and policymakers. Although the 1983 report “A Nation at Risk” offered an opportunity to rethink American education, he said, American students still lag behind their counterparts abroad. The result has been growing concern about teachers as a cause of that decline. But while the literature does show that teachers have a big impact on what students learn, said Marx, policymakers tend to forget other factors, such as infrastructure, curriculum and assessment models.
Now the federal government is getting involved. The 2008 Higher Education Opportunity Act gives the government statutory authority for regulating teacher education. Title II requires institutions with teacher education programs receiving federal funds to send annual reports to the secretary of education; these reports include information about closed, poor-performing or at-risk programs. Title IV, which includes most federal student aid, requires recipients to be enrolled in high-performing teacher education programs. The Department of Education wanted to link those two titles, prompting a negotiated rule-making process.
It turned out that the secretary of education was concerned because only a small number of the 1,500 teacher education programs ever close or receive poor or at-risk ratings. The assumption? Institutions are not exercising quality assurance, so the government should step in. The Department of Education wanted to add more data reporting to Title II, including customer satisfaction, employment outcomes, observations of professional practice and “value-added” data from students of program completers. The Department of Education then wanted to link Titles II and IV and only give TEACH grants to students enrolled in programs that receive the highest ranking. That would be an unprecedented move that could lead the government to link financial aid to quality definitions across all of higher education, Marx warned.
Marx was nominated to be part of the negotiated rule-making process, serving as a non-voting alternate. Because it was a consensus process, all voting members had to agree on all 10 issues put forward. Two groups emerged: a teacher education caucus, and another caucus comprising advocacy groups like Teach for America. “My politics tend to be fairly liberal, but I became a conservative in this group,” said Marx, adding that he believes that teacher education needs improvement and hoped the rule-making process would result in greater accountability. Instead, he said, he lost faith in the whole process, thanks in part to the inadequate data being used to make decisions. In the end, the group did not achieve consensus. As a result, the Department of Education will write its own regulations and go through the normal regulatory process — a process that will be delayed until after the election.
Why quality matters
Jane Halonen, PhD, dean of the College of Arts and Sciences at the University of West Florida, then shifted the attention to improving the quality of undergraduate psychology majors. Halonen chaired the group that produced “APA's Guidelines for the Undergraduate Psychology Major,” which APA approved in 2007. Now she's leading the effort to revise those guidelines in the face of growing concern by legislators, the media, and others that the psychology major is too popular, too easy, and worthless when it comes to getting a job. “That was a call to arms,” said Halonen.
Psychology is indeed popular, she said, noting that psychology is “the most popular major ever.” It's not easy. It doesn't divert students away from science, technology, engineering and mathematics (STEM) majors but is itself a STEM discipline. It also provides a refuge to students who discover that their original major doesn't suit them while psychology speaks to them. And psychology majors do get jobs. But while the discipline has done a good job preparing students for grad school, said Halonen, it hasn't done a good job explaining why the psychology major is good preparation for the workforce. That's why workforce issues occupy a central place in the revised guidelines, which Halonen calls “Guidelines 2.0.”
The revised guidelines reduce the number of learning goals from 10 to five, including professional development. Designed to prepare students for the workforce, graduate school or professional school, this learning goal includes such skills as applying psychological content and skills to career goals, learning self-efficacy and self-regulation, refining project management and teamwork skills, and developing a meaningful direction for life after graduation. The other learning goals focus on the knowledge base, scientific thinking and critical thinking, ethical and social responsibility, and communication.
Halonen ended with a request for feedback (PDF, 1.3MB) on the draft guidelines.
Quality improvement: An important opportunity for psychology
Bruce L. Bobbitt, PhD, vice president for quality management and improvement at Optum® Behavioral Solutions at UnitedHealth Group, then described the growing importance of quality improvement efforts in health care and beyond. While quality is something that is universally desired, said Bobbitt, it is not universally defined. People often say they know it when they see it, he said, but quality needs to have a clear definition, meet some kind of standard, and be based on a process if it is to be improved. To say it is in the eye of the beholder is no longer good enough, said Bobbitt.
The most important concept in quality improvement is that customers define whether something meets a standard. “In psychology, we tend not to think about the voice of the customer because ‘customer’ implies an unseemly thing — someone buying something,” he said. In business, however, companies go out of business if they do not sell the products people want and so spend a lot of time trying to figure out what customers desire. In areas like health care and education, however, the idea of quality improvement is still new and somewhat controversial. What does quality mean in health care settings? Bobbitt listed several meanings, including meeting standards defined by customers, meeting universally agreed upon standards within a professional domain, and using a set of processes and activities to achieve an agreed upon standard that reduces variation. Quality assurance means checking something against a standard, he said, citing accreditation as one example. Quality improvement is a process, he added.
Some of this will be controversial in psychology, Bobbitt admitted. The idea of reducing variation and achieving a standard, for instance, will be difficult in a discipline that embraces variations in the form of different schools. The question of who will define standards in psychology is another unanswered question. Bobbitt ended with a look at challenges and opportunities when it comes to quality and psychology, posing a series of questions. First, is there agreement that something needs to be improved? Is there a market for spending time training people in quality improvement techniques? What is the desired outcome, and is there a process for getting there? What are barriers to achieving the goal? How will success be measured? And what are the costs and return on investment?
Bobbitt concluded by urging participants to think carefully but act quickly as they incorporate quality improvement into psychology, especially given the external environment. In education, he noted, there is a lot of discussion about the need for the profession to establish standards for itself rather than have them imposed from the outside. “I can attest that in health care, we would have preferred that, too, but once the government gets a toe in the regulatory apparatus, you lose control of your process,” he said.
Moving beyond the cultural competence conundrum
The focus then shifted to the challenges of cultural competence.
“How many of you here are culturally competent?” asked Kermit A. Crawford, PhD, director of the Center for Multicultural Mental Health and the Center for Multicultural Training in Psychology at the Boston University Medical Campus. “I didn't raise my hand either.” While cultural competence sounds like the right thing to do, he said, there are some significant problems with the term. Although cultural competence was first described by Stanley Sue, PhD, and others 30 years ago, said Crawford, there is still no consensus about an operational definition. There are no reliable outcome measures, so people rely on what Crawford called “notoriously unreliable” self-reporting. And despite three decades of efforts, significant ethnic and racial disparities remain.
Crawford then identified three major challenges with cultural competence:
Combining the definitions of competence and culture results in an impossibly broad definition of cultural competence: the state of being well-qualified to provide effectiveness across the totality of socially transmitted behavior patterns, arts, beliefs, institutions and all other products of human thought and work.
There is a numbers problem. Race and ethnicity, culture, gender, sexual orientation, socioeconomic classification, age, disability and religion are the classic “big eight” of diversity, said Crawford, adding that those factors can come in combinations. “If you're culturally competent, at least theoretically you have to be competent in working with more than 40,000 different possibilities that could come in the door one person at one time,” he said.
The notion that people can always become more culturally competent means that you can never actually achieve competence.
Models of cultural competence are equally problematic, said Crawford, citing as examples Terry Cross's continuum of cultural competence model, the Campinha-Bacote process of cultural competence model, and the Purnell model of concentric circles. While the latter gets a lot of publicity, said Crawford, it is also incredibly complicated. “We're asking people to keep this in mind when a person is sitting in front of them,” he said, adding that these models fail. The research base is similarly weak, with insufficient controls, inadequate rigor, and inconsistent or inconclusive results.
How can psychologists train students and interns to provide the highest quality cross-cultural care then? Organizations should promote the National Standards on Culturally and Linguistically Appropriate Services from the U.S. Department of Health and Human Services, which offer recommendations for how health care organizations can best serve an increasingly diverse population. Programs should consider the diversity of faculty and students, promote regular discussion of cultural issues, and ensure the curriculum reflects cultural differences. Individuals should engage in soul searching and make sure they are culturally humble, aware and knowledgeable.
When it comes to training students and interns in how to provide culturally competent interventions, said Crawford, the solution may lie in a “culturally adaptive” approach to care. One literature review found that interventions targeted to a specific cultural group were four times more effective than controls and that interventions in clients' own language were twice as effective. The process of adapting psychotherapy includes generating knowledge in collaboration with stakeholders, integrating that with empirical evidence and clinical experience, reviewing plans for culturally adapted interventions with stakeholders, and testing interventions. “That gives us a yardstick we can measure progress by,” Crawford concluded.
How blind spots influence ethical decision making
Next, Ann Tenbrunsel, PhD, the Rex and Alice A. Martin Professor of Business Ethics and co-director of the Institute for Ethical Business Worldwide at the University of Notre Dame, described the ethical challenges that can affect quality.
Despite well-intentioned efforts, people and organizations continue to behave unethically, said Tenbrunsel. That's partly because such efforts take the wrong approach. “Traditionally, ethics was studied from a normative approach — how people should behave,” she said. While that's important, she said, it's not enough. “Before we can get there,” she said, “we need to understand how people actually behave and how they behave in a way that's contrary to their own ideals.” Known as behavioral ethics, this field seeks to understand how people respond to dilemmas in ways that contradict their values — often without realizing they are doing so.
Tenbrunsel outlined four ethical blind spots:
People have biased perceptions of their own ethicality, said Tenbrunsel. These illusions arise because of what happens during the three stages of decision making. When you predict how you'll act in a given situation, she said, your “should self” wins. At the time of the decision, the “want self” wins. While looking back, the “should self” reasserts itself and you see your actions as more ethical than they were. That's because before you act, you're thinking abstractly, but feasibility and desirability concerns come to the fore when it is actually time to act. “It's amazing how easy it is to rationalize our behavior,” said Tenbrunsel. In one study, for example, 80 percent of medical residents said it was unacceptable to take gifts from the pharmaceutical industry. When they were reminded of the sacrifices they were making, however, the number of residents who thought such gifts were acceptable doubled. “Visceral forces,” such as hunger, fatigue and fear, can also prompt unethical behavior.
How people frame a decision also affects how they view it, said Tenbrunsel. One tragic example is the Challenger explosion. When NASA asked the head of engineering to take off his engineering hat and put on a management one, she said, that reframing shifted the decision making from being about safety and product design to being about deadlines and cost. Language can also contribute to ethical fading, she said, citing as examples phrases like “right-sizing” and “creative accounting.”
“The brain is good at paying attention to what it is incentivized to do,” said Tenbrunsel. In one study, for instance, surgical residents eager to practice their skills performed unnecessary surgeries. Social pressure can also drive people to behave unethically.
“We tend not to see the unethical behavior of others when it's not in our best interest to do so,” said Tenbrunsel. When companies are allowed to hire their own auditors, for example, auditors have an incentive to be lax rather than accurate so they will be hired again the next year.
Conference participants spent their first afternoon together in small groups to discuss specific topics and develop questions to explore with the whole group. Chaired by Michael Roberts, PhD, the next day's interactive session focused on the following areas:
Refining expectations for the undergraduate psychology major
This group began by examining the draft of the 2.0 version of APA's “Guidelines for the Undergraduate Psychology Major.” Jane Halonen, PhD, said that the discussion will result in some adjustments. In the knowledge-based section, for example, participants agreed that there was not enough flexibility in some of the exemplars. In the ethical and social responsibility section, participants said the drafters had missed an opportunity to discuss the more positive features of intercultural interaction. And in the professional development section, participants wanted more information about the new kinds of pressures affecting professional development.
The larger group then discussed what they liked best about the proposed guidelines 2.0. Twenty-nine percent thought the revision's enhancement of workforce preparation issues was the best improvement, while another 25 percent cited the reduction of goals from 10 to five. The group also discussed how APA should encourage the adoption of guidelines 2.0. Nearly two thirds said APA should provide formal acknowledgment of programs that adopt the guidelines but not conduct full-fledged accreditation.
Teaching quality improvement
This group began its discussion by reviewing terms and abbreviations commonly used in the quality improvement field. “Very few of us knew what the abbreviations stood for,” said APA staffer Catherine Grus, PhD, who led the group. “That was our first warning sign that this was a topic to pay attention to, not only in how to prepare students but in our own skill set.” Incorporating quality improvement into psychology will also necessitate some cultural shifts, said Grus, citing as one example the emphasis on a “snapshot” in time in psychology research versus the cyclical nature of quality improvement efforts.
More than a third of participants in the larger group strongly agreed that quality improvement has not been consistently included in professional psychology education and training, except within industrial/organizational psychology. Instead, psychologists have often learned these skills outside of formal training experiences. How could quality improvement concepts be integrated into psychology education and training? Almost three-quarters of participants called for a multipronged approach, including collecting patient outcomes and reviewing them during the supervision process; creating storyboards or pictures representing a process, such as a team clinic visit for use as a discussion tool; and encouraging students to measure health cost offset in dissertations focused on health behavior interventions.
Promoting quality in continuing professional development
Led by APA staffer Greg Neimeyer, PhD, this group began by asking the larger group to consider what distinguishes professionals who seem to get stuck in time when it comes to professional growth, development and competence from those who keep up with changes and relish new learning. Most participants agreed that it was not a matter of free time or laziness. Instead, almost two-thirds said that those who keep up have organizations that enable or require them to commit to new learning.
The conversation then turned to the sources participants draw on as they maintain or enhance their own professional development. Almost three-quarters use a combined approach, seeking mentoring from more experienced colleagues, taking courses or workshops in targeted areas of skills building, asking for feedback and consultation with colleagues, and engaging in self-directed study of some kind.
The group then discussed whether APA should have a statement concerning principles of quality continuing professional development or continuing education. More than two-thirds agreed that APA has the potential to be a standard-bearer in relation to defining excellence in psychology, which extends to all phases of learning.
Accountability lessons from teacher education
This group, led by APA staffer Rena Subotnik, PhD, consisted of people concerned about growing public questions about the process and outcomes of teacher preparation, psychologist preparation, and the teaching of psychology at the high school and undergraduate levels. “There was agreement among the group that accountability is a political reality and that we must and should face it squarely,” said Subotnik. “We also agreed that measures and criteria for quality that APA identifies should be widely and publicly known and should be based on data and embellished with powerful examples and stories.” That group also agreed that psychologists and psychologists in teacher preparation should work together closely to enhance effectiveness and share their expertise and ideas.
The larger group then discussed whether graduate programs and internships that prepare professional psychologists should be held accountable for the quality of their graduates. Eighty-four percent of participants agreed that programs should be accountable to funders, the public and the profession.
Participants next discussed what measures would be acceptable to meet public expectations if there were growing public pressure for accountability regarding the quality of graduates, as has happened in teacher education. Almost two-thirds of respondents called for a multitiered approach, including collecting job satisfaction data from graduates, supervisors and employers; having supervisors and employers observe graduates in the field; and measuring improvements in patient health.
Education advocacy awards luncheon
The awards luncheon is a highlight of every ELC. The awards recognize psychologists and other individuals who are advancing psychology education and training through their advocacy activities and support. This year, BEA Chair Michael Roberts, PhD, ABPP, and APA President Suzanne Bennett Johnson, PhD, ABPP, recognized three individuals for their outstanding efforts to gain federal support for psychology education and training:
Ronald Rozensky, PhD, received the Education Advocacy APA Member-at-Large Distinguished Service Award in recognition of his work on the Health Resources and Services Administration's Advisory Committee on Interdisciplinary, Community-Based Linkages. He chaired the committee in 2009 and was involved in four reports to Congress and the secretary of the U.S. Department of Health and Human Services, including a groundbreaking report on preparing an interprofessional workforce for primary care.
Two psychologists received Education Advocacy Grassroots Distinguished Service Awards. Patricia Cole, PhD, was a founding member of APA's Federal Education Advocacy Coordinator grassroots network, and is still recruiting campus training representatives even after stepping down as a coordinator last year. Linda Forrest, PhD, was recognized for her invaluable assistance as APA's government relations staff worked to get the Garrett Lee Smith Memorial Act reauthorized.
Matthew Fitzgerald, chair of the American Psychological Association of Graduate Students, gave Roberts a special citation for his role in getting APA to agree to spend $3 million to help unaccredited internship sites achieve accreditation.
At a reception for Rep. Rosa DeLauro, D-Conn., Cynthia Belar, PhD, gave the congresswoman an award recognizing her commitment to the Graduate Psychology Education Program as a way of meeting the needs of underserved populations.
Graduate Psychology Education (GPE) trainees at a community health center
The conference's final plenary session focused on the role that the GPE Program plays in serving the underserved and training the next generation of psychologists, using one GPE site as an example.
“Like many other health care agencies, we're changing our practice in order to accommodate expectations under the Affordable Care Act,” said Kathy Grisham, executive director of the Community Clinic at St. Francis House. “What we're trying to implement is a patient-centered medical home.”
The GPE Program is helping the Community Clinic achieve that goal. Ana Bridges, PhD, an assistant professor of psychology at the University of Arkansas, described how the GPE program works at her site. GPE funding has allowed the program to add didactic training on diversity, medical consultation and integrated care; create a month-long immersion program in collaboration with a Puerto Rican medical school; and encourage students to engage in political advocacy. The “capstone” of the GPE Program is a 10-month practicum at the Community Clinic, which gives students a chance to experience firsthand the GPE's public health approach to health care.
“In traditional psychology, the emphasis is on effective treatment,” said Bridges. “The public health perspective is looking at impact, so effectiveness is one but not the only part of the equation.” If a more effective treatment requires individuals to visit the clinic for 20 weeks, for instance, it won't have as big an impact as a less effective intervention that can be more broadly disseminated.
That approach is working at the Community Clinic. Evaluations show reductions in patients' distress, even though the average is just 1 and a half visits per patient. “What I tell Congress and state representatives when I meet them is that we get 75 percent of the therapeutic effect of a 20-visit treatment in 1 and a half visits,” said Bridges. Of course, students benefit, too. An evaluation has found that the longer GPE trainees are with the program, the more they report they feel ready to work in integrative settings, consult with medical providers, and practice evidence-based psychology.
Debbie Gomez, a clinical psychology doctoral student at the University of Arkansas, then shared her experience as a GPE trainee. The didactic training has taught her new skills, such as reducing some treatments to just 20-minute interventions. And the practicum is such a rich training experience, said Gomez, that she has signed on for an additional year. By working alongside physicians, nurses and other medical professionals to provide care to each patient, she explained, she is learning how to be an effective interprofessional collaborator. For one young patient, a boy from an abusive home, Gomez's work teaching the boy's foster parents behavior management strategies in one session and good sleep hygiene in another turned the child around so much that the foster parents now plan to adopt him. Another GPE trainee provided psychoeducation to a woman with panic attacks and high emergency room use.
On the conference's last day, participants headed to Capitol Hill to visit their congressional representatives and advocate for psychology's educational priorities. Participants made 175 visits to the offices of their members of Congress to advocate for the GPE Program.