Goal 1: Knowledge Base of Psychology
Demonstrate familiarity with the major concepts, theoretical perspectives, empirical findings and historical trends in psychology.

1.1:

Characterize the nature of psychology as a discipline.

1.2:

Demonstrate knowledge and understanding representing appropriate breadth and depth in selected content areas of psychology: theory and research representing general domains, the history of psychology, relevant levels of analysis, overarching themes, and relevant ethical issues.

1.3:

Use the concepts, language, and major theories of the discipline to account for psychological phenomena.

1.4:

Explain major perspectives of psychology (e.g., behavioral, biological, cognitive, evolutionary, humanistic, psychodynamic, and sociocultural).


ASSESSMENT CATEGORY

ESTIMATE OF METHOD'S POTENTIAL

CLASSROOM/COURSE DATA

Overall Strong potential. Classroom activities and course data naturally provide venues in which to assess content knowledge in psychology. However, current assessment trends suggest that concentrating solely on these indices may not provide sufficient information to provide meaningful feedback on program integrity.

OBJECTIVE TESTS

Strong potential. Despite the inherent difficulties in constructing valid and reliable objective tests, the method can effectively assess content learning. However, most objective tests tend to evaluate student knowledge more routinely at lower levels of thinking (e.g., rote, simple application).

ESSAY TESTS

Strong potential. Despite the labor intensiveness of providing feedback on essay tests, this approach facilitates greater access to measuring deeper levels of content learning. Faculty are increasingly turning to the use of rubrics and specified criteria to address problems of reliability in grading.

EMBEDDED QUESTIONS AND ASSIGNMENTS

Strong potential. Choosing some courses in which program assessment activities can occur can still facilitate specific assessment of content knowledge. For example, embedding a departmental assessment of ability to demonstrate APA format in a methods class provides a a reasonable vehicle for assessing content knowledge of APA format. Other emphases are possible in embedded assessments, including ethics, persistent themes, or historical detail among others.

CLASSROOM ASSESSMENT TECHNIQUES

Strong potential. Faculty engage in classroom assessment techniques to provide spot checks of how well students are learning specific concepts. Although the focus is understandably narrow (e.g., the content of a particular class), the method provides optimal feedback for the faculty member concerned with what students are learning and retaining.


INDIVIDUAL PROJECTS/PERFORMANCE ASSESSMENT

Overall Strong potential. The knowledge base of psychology is predictably the foundation for most individual projects. These generally offer the advantage of studying some corner of the discipline in depth; however, breadth of exposure to content may be a casualty given the time limitations most faculty and students face.

WRITTEN PRODUCTS

Strong potential. When assignments are well-designed, written products should provide insight into what students know and don't know about content. Faculty have discovered that specifying how much content (e.g., number of required references) may facilitate the depth of exploration the faculty member had in mind when designing the project instructions.

ORAL PRESENTATIONS

Strong potential. oral presentations also provide insight into student learning of the content. In addition, the opportunity to engage students in questions allows faculty and classmates to probe the depth of student knowledge while building oral communications skills.

GRAPHIC TESTS AND DISPLAYS

Strong potential. Concept maps can reveal the nature of associations that students develop regarding specified content in the discipline.

POSTERS

Strong potential. posters can provide a more global sense of what students understand due to the brevity of the medium. However, informal questioning can fill in the gaps about what students have not communicated in the poster.

STRUCTURAL/SITUATIONAL ASSESSMENTS

Strong potential. Although situational assessments tend to emphasize application of learning, applications are built on a disciplinary foundation. The success of assessment of content learning will depend on the expertise demonstrated in the design of the applied assessment.


SUMMATIVE PERFORMANCE ASSESSMENT

Overall Mixed potential. Assessments that occur at the end of a program vary in their effectiveness for assessing content. In some cases, depth of knowledge required by some demonstrations will not allow an estimate of broad knowledge in the discipline.

STANDARDIZED TESTS

Mixed potential. Currently there are two primary standardized exams that allow for comparison across institutions as well as tracking changes in program achievement over time: the Academic Concentration Applied Test (ACAT) and the Major Fields Test by ETS. Each exam measures knowledge in the subdisciplines of psychology, but student course selection may adversely affect overall performance on either instrument. Care must be exercised in interpreting the results.

LOCALLY DEVELOPED TESTS

Strong potential. Developing a department examination is a time-consuming but effective way to track changes in student knowledge over time but does not provide normative comparison with other programs. In addition, test security and changes in content knowledge make this practice complex.

CAPSTONE EXPERIENCES

Mixed potential. The utility of capstone performance to assess content knowledge greatly depends on the scope of the course design. Students are more likely to develop deep levels of expertise in more narrowly defined areas of psychology in most capstone designs. To the extent that their performance represents what they can do within specific performance parameters, the capstone may be a satisfying method to assess the ability to deal with content in sophisticated ways. However, more broadly conceived capstone courses (e.g., history, systems of psychology) may provide broader assessment opportunities.

INTERNSHIP/PROFESSIONAL APPLICATIONS

Moderate potential. Internships and professional applications facilitate specific types of applications. For example, an industrial-organizational internship may be an optimal way for a student to demonstrate the knowledge base related to the subdiscipline, but it may not be satisfying as a broad assessment.

PORTFOLIOS

Strong potential. Selecting and justifying selections from explicit departmental criteria will facilitate student reflection regarding the level of expertise they have developed in the content of psychology.

ASSESSMENT CENTER METHODS

Strong potential. Similar to structured assessments, the in-basket strategies of assessment center methods can provide insight into student abilities to apply principles from the content of psychology.

CASE AND LONGITUDINAL STUDIES

Limited potential. Because much of content learning is fragile, longitudinal studies of content retention are likely to be disappointing sources of student learning. In addition, merely reporting how sturdy content learning is over time rather than directly assessing may be content learning will be a less reliable measure.


SELF-ASSESSMENT

Overall Mixed potential. As can be seen from student anticipation of how well they performed on an exam, student ability to judge their own expertise is variable.

STUDENT JOURNALS

Moderate potential. Journal instructions can specify the manner in which students should interact with the content of the discipline. For example, instructions might require that students demonstrate the appropriate application of five concepts or principles from the discipline. Students will vary in their own expert judgment on the success of addressing the concepts or principles in the manner anticipated by the faculty.

SELF-CRITIQUES

Mixed potential. Students have limited experience in making judgments about how well they have met the content criteria of a given assignment. Students often drift to the easier-to-judge aspects of performance, such a format concerns, interest generation, or comfort level rather than exploring how well they have reflected content expertise.


COLLABORATION

Mixed potential. Some methods offer effective avenues for examining content and theory, while others are less promising.

RESEARCH TEAMS

Strong potential. Research teams give students an opportunity to demonstrate content expertise in two dimensions: research methods and the subject matter that the research is designed to explore. Students can receive separate evaluations on the extent to which they have collectively demonstrated research expertise as well as whether they have appropriately represented the target content.

GROUP PROJECTS

Strong potential. Group projects can also provide a content-based opportunity to develop group skills. Projects can reflect successful or unsuccessful strategies to master relevant content and principles. However, group projects suffer similar limitations to individual projects. Committing in-depth study to one arena may require the sacrifice of exposure to other content in the course.

ON-LINE ACTIVITIES

Limited potential. Unless students are given very constrained instructions regarding how to pursue content collaboration, the use of chat room or email exchanges to monitor content expertise may be challenging.


INTERVIEWS & SURVEYS

Mixed potential. In general, surveys and interviews are not recommended because the assessment of content is not likely to be direct.

SATISFACTION SURVEYS

Poor potential. Assessing content expertise through satisfaction surveys is too indirect to be recommended.

PERFORMANCE REVIEWS

Moderate potential. Although this method is time consuming, the next step (e.g., grad school or employment) can provide for direct observation of the content of psychology.

EXIT INTERVIEWS

Limited potential. Exit interviews tend to focus on affective dimensions of learning as well as the collection of impressions that may facilitate program improvement. Content mastery is not routinely the focus of exit interviews.

EXTERNAL EXAMINER INTERVIEWS

Moderate potential. A rigorous external examiner protocol could focus on the depth of content mastery of individual students. However, the expense and time limitations of this approach tend to focus on other aspects of student performance.

FOCUS GROUPS

Not recommended. Focus groups typically convene to solve a specific problem rather than provide a measure of content mastery. Such academic development may be inferred but there are other more direct methods to assess mastery.

FOLLOW-UP ALUMNI INTERVIEWS

Not recommended. Engaging with alumni over the specifics of content that they can recall is likely to be a discouraging assessment strategy since the detail of the discipline dims with distance from graduation.


ARCHIVAL MEASURES

Mixed potential. Archival measures can provide some insight into the content bases to which students have been exposed but will do little to assess more formal learning of the content in the discipline.

TRANSCRIPT ANALYSIS

Limited potential. Individual transcript analysis can provide not just a measure of the various content bases to which the student has been exposed but through grades can provide a gross measure of achievement in those areas. However, assessment experts recommend that other noncourse-based strategies will be more effective in providing legitimate measures of student and program achievement.

ANALYSIS OF TRANSFER PATTERNS

Limited potential. Examining patterns of what transfer students provide can help departments determine what and when to offer in the curriculum, but will shed little light on the quality of learning.

SYLLABUS AUDIT

Limited potential. A syllabus audit can isolate the range of content exposure that students experience but will be poor indicators of actual learning.

DEMOGRAPHIC DATA ANALYSIS

Not recommended. Understanding the characteristics of the student body will provide little insight into their content mastery.

ALUMNI DATABASE

Not recommended. The alumni database does not directly reveal student expertise in the content of psychology. However, many programs rely on the percentage of students who go on to graduate school in the area as an indirect measure of content expertise.

LIBRARY STATISTICS USAGE/WEB HITS

Not recommended. Content expertise is not apparent in this archival analysis.

Goal 2: Research Methods in Psychology
Understand and apply basic research methods in psychology, including research design, data analysis and interpretation.

2.1

Describe the basic characteristics of the science of psychology.

2.2

Explain different research methods used by psychologists.

 
  1. Describe how various research designs address different types of questions and hypotheses
  2. Articulate strengths and limitations of various research designs
  3. Distinguish the nature of designs that permit causal inferences from those that do not

2.3

Evaluate the appropriateness of conclusions derived from psychological research.

 
  1. Interpret basic statistical results
  2. Distinguish between statistical significance and practical significance
  3. Describe effect size and confidence intervals
  4. Evaluate the validity of conclusions presented in research reports

2.4

Design and conduct basic studies to address psychological questions using appropriate research methods.

 
  1. Locate and use relevant databases, research, and theory to plan, conduct, and interpret results of research studies
  2. Formulate testable research hypotheses, based on operational definitions of variables
  3. Select and apply appropriate methods to maximize internal and external validity and reduce the plausibility of alternative explanations
  4. Collect, analyze, interpret, and report data using appropriate statistical strategies to address different types of research questions and hypotheses
  5. Recognize that theoretical and sociocultural contexts as well as personal biases may shape research questions, design, data collection, analysis, and interpretation

2.5

Follow the APA Code of Ethics in the treatment of human and nonhuman participants in the design, data collection, interpretation, and reporting of psychological research.

2.6

Generalize research conclusions appropriately based on the parameters of particular research methods.

 
  1. Exercise caution in predicting behavior based on limitations of single studies
  2. Recognize the limitations of applying normative conclusions to individuals
  3. Acknowledge that research results may have unanticipated societal consequences
  4. Recognize that individual differences and sociocultural contexts may influence the applicability of research findings

ASSESSMENT CATEGORY

ESTIMATE OF METHOD'S POTENTIAL

CLASSROOM/COURSE DATA

Mixed potential. Useful for providing assessment of factual knowledge and some limited application. Research skills will be better assessed with other strategies that involve activities outside the traditional classroom.

OBJECTIVE TESTS

Moderate potential. Good method for assessing primarily factual knowledge, especially if the test questions can be demonstrated to have strong psychometric properties. Less useful in assessing application and higher level comprehension such as designing original research or performing and interpreting statistical calculations.

ESSAY TESTS

Moderate potential. More powerful method for assessing application and higher level comprehension, but is still limited in its ability to assess ability to design original research or perform and interpret statistical calculations because of limited time frame in testing situation.

EMBEDDED QUESTIONS AND ASSIGNMENTS

Strong potential. Because research methods courses are often a prerequisite for advanced courses, departments may also dictate specific assignments (e.g., research projects) that should be embedded in required coursework across different sections of the same methods and/or statistics courses and in subsequent courses in the curriculum.

CLASSROOM ASSESSMENT TECHNIQUES

Moderate potential. Provides quick, but often limited assessment, on student understanding and performance.

INDIVIDUAL PROJECTS/PERFORMANCE ASSESSMENT

Strong potential. Faculty can interpret sophistication in research skills from intellectual products. Because projects are done outside class, there may be some risk that a student's work is not an individual production. However, this limitation can be addressed by including a reflective piece that assesses the targeted skills.

WRITTEN PRODUCTS

Strong potential. Written reports of original research provide an ideal context for assessing the methodological skills and ethical issues involved in designing, conducting, and evaluating the results. Time constraints (e.g., IRB approval) and the labor intensive nature of original research may limit usefulness in some courses. Research projects may also only assess an understanding of the particular methods used, not a broader understanding.

ORAL PRESENTATIONS

Moderate potential. Individual oral presentations provide some opportunity to evaluate quality of research skills and ability to present a shorthand summary. However, these may be challenging to judge in the moment and they often lack details that allow for in depth assessment.

GRAPHIC TESTS AND DISPLAYS

Moderate potential. Graphical presentations can be useful in mapping the research process. Statistical understanding can be assessed through the accuracy and clarity of graphical presentations.

POSTERS

Moderate potential. The limited space available in most posters may not provide an ideal context in which to evaluate the full understanding and application of research methods.

STRUCTURAL/SITUATIONAL ASSESSMENTS

Strong potential. The presentation and interpretation of research findings in the popular media can be used to have students demonstrate their skill in addressing issues related to the design and interpretation of research. Current events can also be used as a starting point for students to design and conduct original research projects.

SUMMATIVE PERFORMANCE ASSESSMENT

Mixed potential. Strategies in this category range from poor to strong.

STANDARDIZED TESTS

Moderate potential. Although standardized tests assess factual knowledge related to research methods and statistics, they fail to evaluate application of skills at the level identified for these outcomes.

LOCALLY DEVELOPED TESTS

Moderate potential. Like standardized tests, they primarily focus on factual knowledge as opposed to application. In addition, they may lack strong psychometric properties.

CAPSTONE EXPERIENCES

Strong potential. Assuming that the capstone course or project has an expressive requirement (e.g., writing or speaking), it can provide an integrated demonstration opportunity.

INTERNSHIP/PROFESSIONAL APPLICATIONS

Limited potential. The focus of most applied internship experiences tends to be in applied areas of psychology. However, some experiences may include the opportunity to design, conduct, and evaluate research (i.e., a research internship).

PORTFOLIOS

Strong potential. Explicit criteria that ask students to select "works" based on what these reveal about their research skills can provide an opportunity to evaluate the evolution of their abilities through a focused reflection on why they selected the items they did.

ASSESSMENT CENTER METHODS

Poor potential. Assessment center methods are generally limited in time and focus. They are unlikely to provide in depth information on all the outcomes associated with this goal because of inherent time constraints.

CASE AND LONGITUDINAL STUDIES

Limited potential. The complexities of tracking all of these skills over time mitigates against adopting this strategy to monitor their evolution, especially where different curses in the curriculum vary in the requirement to use these skills. Programs requiring both a survey research methods course with an original research project and a capstone research experience may have a limited opportunity to evaluate longitudinal development.

SELF-ASSESSMENT

Mixed potential. Developing students' metacognition of their understanding of research methods has seldom been addressed self-assessment strategies.

STUDENT JOURNALS

Moderate potential. Although it may be unlikely that faculty would choose to invest time reading about students' struggle to learn research methods, this technique can be adapted to a research journal where students keep a record of research ideas, development and progress that reflects application of research methods knowledge.

SELF-CRITIQUES

Limited potential. When faculty can specify relevant performance criteria, students can provide an estimate of their research and statistical skills.

COLLABORATION

Moderate potential. Techniques in this category are moderate to limited in usefulness.

RESEARCH TEAMS

Moderate potential. Research teams can develop and evidence expertise in research skills through peer involvement and often model the collaborative nature of research at the professional level. Unfortunately, research teams may reduce a beginning or weak student's direct involvement in generating research ideas, research design, statistical analysis, and interpretation of results.

GROUP PROJECTS

Moderate potential. Group projects involve similar issues to those of research teams.

ON-LINE ACTIVITIES

Limited potential. Archived on-line chat rooms, listservs, or bulletin boards can provide opportunities to assess the development and evolution of research ideas from start to finish.

INTERVIEWS & SURVEYS

Mixed potential. The assessment of attitudes by the students or other stakeholders may provide some feedback about research methods and statistical competence, but attitudes may not be an accurate indication of true skill attainment.

SATISFACTION SURVEYS

Strong potential. The abilities of students to design, conduct and evaluate research can be evaluated by employers, graduate advisors, or other stakeholders. External evaluators may explicitly need to be prompted to address these skills. This may be particularly effective for those students who continue in graduate programs in psychology.

PERFORMANCE REVIEWS

Strong potential. Stakeholders can provide an estimate of strengths and weaknesses within research skills with appropriate prompts for reflection.

EXIT INTERVIEWS

Moderate potential. Students can be asked to reflect on the evolution of their research and statistical skills.

EXTERNAL EXAMINER INTERVIEWS

Limited potential. External examiners can ask probe questions to evaluate student comfort levels about research and statistical skills, but the evaluation of self-report relative to actual performance quality may be problematic unless evaluators also review actual products.

FOCUS GROUPS

Limited potential. Although focus groups most often convene to solve specific departmental problems, this area is often core to a program and challenging to students and may be more likely to be addressed in this context.

FOLLOW-UP ALUMNI INTERVIEWS

Limited potential. Although a follow-up interview on this topic might invite demand characteristics, it may be useful to assess perceived skill levels in post-graduate settings.

ARCHIVAL MEASURES

Limited potential.

TRANSCRIPT ANALYSIS

Moderate potential. Transcript analysis can reveal the pattern of courses students may engage in (or avoid) in the development and use of research method s and statistical skills. The transcript analysis can provide both patterns and some in-class estimates of quality of student performance although the value of these may be limited.

ANALYSIS OF TRANSFER PATTERNS

Limited potential. Departments can benefit by understanding the transfer courses that students may have taken in research methods and statistics and making comparisons to students who took departmental courses in these areas.

SYLLABUS AUDIT

Moderate potential. An analysis of which courses include content or projects emphasizing research methods or statistics may be a helpful first step in diagnosing where these skills need to be enhanced.

DEMOGRAPHIC DATA ANALYSIS

Not applicable.

ALUMNI DATABASE

Not applicable.

LIBRARY STATISTICS USAGE/WEB HITS

Not applicable.

Goal 3: Critical Thinking Skills in Psychology
Respect and use critical and creative thinking, skeptical inquiry and, when possible, the scientific approach to solve problems related to behavior and mental processes.

3.1

Use critical thinking effectively.

3.2

Engage in creative thinking.

3.3

Use reasoning to recognize, develop, defend, and criticize arguments and other persuasive appeals.

3.4

Approach problems effectively.

ASSESSMENT CATEGORY

ESTIMATE OF METHOD'S POTENTIAL

CLASSROOM/COURSE DATA

Overall Mixed potential. Classroom and course data can be used to assess critical and creative thinking, but the quality of the assessment depends on what is measured in these settings and not the setting per se.

OBJECTIVE TESTS

Limited potential. Objective tests can be used to assess critical thinking, but good objective tests of critical thinking are difficult to construct and cannot be used to assess what students actually do in an unstructured setting where critical thinking is required. They cannot assess the propensity to engage in critical thought. They are better as measures of recognition memory, and hence of limited usefulness in assessing critical thinking.

ESSAY TESTS

Strong potential. An essay test that poses an ecologically-valid scenario (ideally somewhat complex) where students need to explain/ describe their thinking and the conclusion they reached or problem they solved can be a good way to assess critical thinking.

EMBEDDED QUESTIONS AND ASSIGNMENTS

Strong potential. An embedded question or assignment can provide a measure of student's propensity to think critically (i.e., do they engage in critical thinking when the need for critical thinking is not cued or labeled).

CLASSROOM ASSESSMENT TECHNIQUES

Poor potential. Classroom assessment techniques can include reflections on what was learned. It is more likely useful as feedback to instructors about what students believe they have learned than a measure of learning per se.

INDIVIDUAL PROJECTS/PERFORMANCE ASSESSMENT

Overall good potential. Individual projects and performance assessment can be good measures of critical thinking, assuming that the project calls for extended and careful thought. The nature of the project or performance (e.g., solve a novel problem) is what determines the quality of the assessment. The quality of a critical thinking assessment most often lies in the way the instructor crafted the assignment and explained it to students.

WRITTEN PRODUCTS

Strong potential. Like essay tests, a written project needs to allow the student to show the thinking process that went into a conclusion or a solution to a problem.

ORAL PRESENTATIONS

Strong potential. oral presentations are just an alternative format for presenting one's thinking, and thus are similar to written products in their ability to assess critical thinking.

GRAPHIC TESTS AND DISPLAYS

Strong potential. A graphic display of one's thinking can be an excellent assessment of the quality of a student's thinking. There are many concept maps and other ways to map verbal information onto spatial arrays that are well suited for critical thinking assessment. A completed template that shows the parts of a persuasive argument, for example, can be used to clarify complex topics and provide a "picture" of the student's thinking.

POSTERS

Strong potential. A poster can, and probably should, contain a mix of verbal and graphic displays. It can be used to assess critical thinking, if the topic or reason for the poster requires critical thought.

STRUCTURAL/SITUATIONAL ASSESSMENTS

Strong potential. When this category includes activities such as role-playing, seeing problems from multiple perspectives, and similar activities, it can be a good way to demonstrate critical thinking skills.

SUMMATIVE PERFORMANCE ASSESSMENT

Overall Mixed potential. Summative assessments usually refer to tests that are normed to provide comparative data, usually at the completion of a program of study. The normative information can be useful, but only if the test is valid in that it relates to the way students think critically when they are not in class.

STANDARDIZED TESTS

Moderate potential. There are several standardized tests of critical thinking. Unfortunately, they tend to be multiple-choice tests with short problems or scenarios described in each question. They are not generally good measures because real life is much messier, and there is rarely a single correct answer to ill-defined problems. They also do not measure what student's actually do in less structured settings. A quality standardized test is possible, if it includes both constructed response and multiple-choice alternatives to show how students approach problems and whether they can recognize a good response when they have to select from among a small set of alternatives.

LOCALLY DEVELOPED TESTS

Limited potential. A locally developed test can reflect the curriculum that is taught, so it can be more useful to instructors, but locally-developed tests will rarely have the psychometric properties of good reliability and validity that a standardized test will have.

CAPSTONE EXPERIENCES

Strong potential. A capstone experience can be an extended project that requires the application of critical thinking skills to a wide range of issues. If well designed, capstone experiences can provide meaningful measures of critical thinking, but the instructor needs to have clear critical thinking objectives in mind when planning the capstone experience because it is not likely to assess critical thinking without deliberate planning.

INTERNSHIP/PROFESSIONAL APPLICATIONS

Limited potential. Ideally, an internship or other professional application would require evidence of critical thinking, but they would need to be designed with this specific outcome in mind and that rarely happens. In general, instructors do not have direct control over internship experiences, which can make the value of an internship or other application as an assessment of critical thinking hit-or-miss.

PORTFOLIOS

Strong potential. A portfolio that is well planned to show growth in critical thinking skills over time (e.g., four years in college) can provide a good index of gains in critical thinking. Like the other methods, the value of portfolios depends on how carefully the intended critical thinking outcomes are articulated and carried across several different courses.

ASSESSMENT CENTER METHODS

Strong potential . An assessment center is usually a single place where a variety of assessment activities are planned and data are collected. Often they will include simulations of real-life scenarios and problems. If the activities are well planned, they can provide valuable data about critical thinking (e.g., an in-basket exercise).

CASE AND LONGITUDINAL STUDIES

Limited potential. Following students' development in critical thinking over time may be somewhat challenging in the absence of objective measures of that growth.

SELF-ASSESSMENT

Overall Mixed potential. A self-assessment is a student's own analysis of how well he or she is thinking. We know from a large number of studies that most people are poor judges of how well or how critically they think. These techniques can be useful over time if students learn to be more accurate in their self-assessment, but there are little data to suggest that this actually happens.

STUDENT JOURNALS

Mixed potential. Student journals seem to work well for a subset of students who actually reflect on their thinking. For many students, they are worthless exercises in filling up paper. Some instructors like to give students the experience of student journals so that those students who benefit from this activity are not penalized by those who do not. It can be useful, but only for some students. Clear instructions for journaling are required.

SELF-CRITIQUES

Limited potential. Given that most people are very happy with their ability to think critically, self-critiques can only be useful if students learn to see their own weaknesses. It is difficult to change belief about how well one thinks, but not impossible. Thus, one outcome of critical thinking instruction is the seemingly paradoxical result that students often rate themselves as poorer thinkers at the end of a course than at the start. This is a positive outcome, but it tells instructors very little about the student's actual ability to think critically.

COLLABORATION

Overall Strong potential. When students collaborate, they think in groups. Because much of the thinking they will do outside of class will involve other people, it can be a valid approach to assess critical thinking.

RESEARCH TEAMS

Strong potential. If the research team is given a problem that requires critical thinking and good guidelines for teamwork are provided so that each team member must contribute to some of the thinking, it can be useful. Instructors will want to capture at least a sample of the group thinking process so that it can be reviewed with each team.

GROUP PROJECTS

Strong potential. Group projects may be designed so that success only can occur when the group engages in effective critical thinking. Groups can process where their critical thinking was faulty to learn from their error.

ON-LINE ACTIVITIES

Limited potential. An on-line collaboration offers the possibility of tracking the thinking process via the written exchanges among team members. Of course, instructors would want students to know that their exchanges are being monitored.

INTERVIEWS & SURVEYS

Overall Limited potential. Interviews and surveys usually ask specific questions about individual beliefs and perceptions. They are not useful in assessing what is learned because they to focus on what students believe they learned and how satisfied they are with the learning.

SATISFACTION SURVEYS

Poor potential. Satisfaction surveys are often called "smilies" because respondents indicate how happy they are with an assignment or course. These are not the same as actual measures of what was learned and cannot be substituted for performance indicators.

PERFORMANCE REVIEWS

Limited potential. Stakeholders can be asked to rate the quality of critical thinking in their evaluations of performance but may require training to understand the parameters being investigated.

EXIT INTERVIEWS

Poor potential. Exit interviews occur when students are leaving a course or program, most often at graduation. They are reflections about what was good or bad about a program of study. Although these measures provide useful data, they usually do not measure critical thinking.

EXTERNAL EXAMINER INTERVIEWS

Moderate potential. External examiners are used to provide an outside (i.e., unbiased) evaluation of the quality of learning. The ability of external evaluators to measure critical thinking depends on what they ask. If they ask satisfaction questions, then they are not assessing critical thinking, but if they ask students to think through a complex problem and explain what they are doing, the assessment can be a measure of critical thinking.

FOCUS GROUPS

Limited potential. Focus groups are often group evaluations of a program or course. They do not provide evidence of critical thinking unless the group is asked to solve a problem, reach a conclusion, make a complex decision or engage in some other critical thinking task.

FOLLOW-UP ALUMNI INTERVIEWS

Moderate potential. Alumni follow-ups tend to utilize satisfaction questions, but they could provide evidence of the long-term retention of critical thinking skills and their transfer to novel domains if the alumni are asked questions that require critical thinking.

ARCHIVAL MEASURES

Overall Poor potential. Archival methods use data that are already available. As in the other categories, the quality of the assessment depends on what is in the available data.

TRANSCRIPT ANALYSIS

Limited potential. A transcript analysis is not likely to tell us much about critical thinking skills because we do not know what was required in each of the classes. Research has shown that much of the learning that occurs in college is relatively low level direct recall of information or low-level inferences.

ANALYSIS OF TRANSFER PATTERNS

Limited potential. Course-taking patterns are not likely to useful by themselves, but could be useful to see how different patterns relate to more valid measures of critical thinking.

SYLLABUS AUDIT

Mixed potential. An audit of a syllabus can sometimes show if critical thinking skills are being taught and learned in a particular class, but most often the syllabus is a list of reading assignments, dates assignments are due, and exam dates. There is rarely any information in the syllabus that provides a clue as to what students are required to do with the information to-be-learned.

DEMOGRAPHIC DATA ANALYSIS

Poor potential. These analyses will not tell us anything about the quality of the thinking of any individual or group.

ALUMNI DATABASE

Poor potential. An alumni data base that provides information about the types of careers that students enter upon graduation and where they are working in 5 to 10 years can allow us make inferences about critical thinking. In general, some careers (e.g., law, research) require better thinking skills than others (e.g., most clerical positions), but this is not a strong assessment method.

LIBRARY STATISTICS USAGE/WEB HITS

Poor potential. Ideally, students who read more should be better thinkers, but we do not know if this hypothesized relationship is true. A better index might be what they chose to read, but this is not a direct measure of critical thinking ability and it requires too many inferences to qualify as a valid assessment.

Goal 4: Application of Psychology
Understand and apply psychological principles to personal, social and organizational issues.

4.1

Describe major applied areas of psychology (e.g., clinical, counseling, industrial/organizational, school, health).

4.2

Identify appropriate applications of psychology in solving problems, such as

4.3

Articulate how psychological principles can be used to explain social issues and inform public policy.

4.4

Apply psychological concepts, theories, and research findings as these relate to everyday life.

4.5

Recognize that ethically complex situations can develop in the application of psychological principles.

ASSESSMENT CATEGORY

ESTIMATE OF METHOD'S POTENTIAL

CLASSROOM/COURSE DATA

Mixed potential. Classroom strategies show variable potential in measuring how students apply the concepts and principles they learn in their psychology courses.

OBJECTIVE TESTS

Limited potential. objective tests (e.g., multiple-choice items) can assess knowledge of the roles of applied areas (e.g., employee selection, training, and evaluation in I/O psychology) and the differences among areas of applied psychology (e.g., clinical and counseling psychology).

ESSAY TESTS

Strong potential. Essay questions can assess knowledge of the application of psychology if they require students to describe examples of how psychological principles and methods can be used to solve specific problems (e.g., decreasing a child's tantrums, strengthening a college student's study skills, or helping an adult overcome a phobia) or how ethical issues can decrease the desirability of some applications.

EMBEDDED QUESTIONS AND ASSIGNMENTS

Strong potential. Embedded assessments represent activities that the department has agreed will provide a good measure of student learning or progress but take place within the confines of a class. Classroom assignments can be used to assess students ability to apply psychological principles, theories, and methods if they are designed to do so. For example, students can apply what they have learned about stress management in an assignment that requires them to (1) identify the major stressors in their lives, (2) devise a plan to improve their ability to cope with these specific stressors, and (3) evaluate the effectiveness of their plan.

CLASSROOM ASSESSMENT TECHNIQUES

Not applicable. Classroom assessment refers to informal methods to determine whether or not students understand course material (e.g., the end-of-class one minute paper). Thus they are more suited to providing feedback to teachers about what is going on in their classrooms than producing data about students ultimate ability to apply psychological principles and methods.

INDIVIDUAL PROJECTS/PERFORMANCE ASSESSMENT

Mostly Strong potential. Nearly all of the methods that address individual performance are reasonable to optimal means of addressing the application of concepts.

WRITTEN PRODUCTS

Strong potential. Any written assignment (e.g., a term paper, moral dilemma analysis) that requires students to describe how they would apply their psychological knowledge would be useful as a means of assessing knowledge of how psychological principles and methods can be applied. Of course, knowing how to apply psychological principles and methods and actually being able to apply them successfully are two different stories.

ORAL PRESENTATIONS

Strong potential. oral presentations can be used to assess the ability to apply psychology in the same way that written products can.

GRAPHIC TESTS AND DISPLAYS

Limited potential. Simple graphic representations may be insufficient with regard to clarifying an application.

POSTERS

Strong potential. A poster can provide substantial evidence of students ability to apply what they have learned in their methods classes. It can also provide faculty with an opportunity to evaluate students ability to "think on their feet" when they are asked questions during a poster session, which provides another venue for demonstrating application skills.

STRUCTURAL/SITUATIONAL ASSESSMENTS

Strong potential. Simulations in which a "real world" situation is created in an artificial environment (e.g., counseling sessions in which the student "counselor" must provide counseling to a fellow student who is role playing a particular DSM category) can provide faculty with a rich opportunity to assess students ability to apply what they have learned in the classroom (e.g., listening skills, the development of rapport, professional mannerisms, etc.).

SUMMATIVE PERFORMANCE ASSESSMENT

Mixed potential. Summative methods must have a predominant focus on application to serve this goal. Some summative approaches tend to have a broader focus.

STANDARDIZED TESTS

Limited potential. There are some nationally standardized tests (e.g., the ACAT and the ETS Undergraduate Field Test) that contain subtests the measure student's knowledge of psychological research methods (e.g., research design, statistical analysis, and graphic interpretation), which are legitimate examples of how psychologists apply the scientific method to solvable problems. However, most question sets favor lower-order questions rather than those that involved applied skills.

LOCALLY DEVELOPED TESTS

Limited potential. A locally developed test will allow faculty the opportunity to collaborate to produce a locally developed test that incorporates application, but producing applied items will be as difficult locally as it is in national exams.

CAPSTONE EXPERIENCES

Optimal potential. There are many types of capstone experiences, but those the require enrollees to "do the discipline" are probably the most effective way for a department to assess its student's ability to apply the psychological principles and methods they have acquired in their previous class work. Capstone classes provide students with an opportunity "to demonstrate comprehensive learning in their major through some type of product or performance" (Palomba & Banta, 1999, p. 124). In other words, a capstone is a class in which senior psychology majors are required to pull together what they have learned in their previous classes and use this integrating experience to demonstrate they are capable of doing what they should be able to do as they graduate from the program (e.g., perform research in a capstone laboratory or demonstrate clinical skills during an internship with a local crisis clinic). This process serves a dual purpose. It allows psychology majors with a final opportunity to practice and demonstrate the skills they will need to succeed after graduation on the job or in graduate school. It also provides the Psychology Department with a final opportunity to assess whether or not it has been successful in its mission to produce psychology majors who are capable of applying what they have learned in their previous seven semesters.

INTERNSHIPS OR PROFESSIONAL APPLICATIONS

Strong potential. See Capstone Experiences above for description of internships as assessment strategies. An internship or practicum taken under the direction of an on-the-job professional can be an invaluable experience for psychology students and it can also provide quality feedback to a department about its students' ability to apply what they have learned in the classroom if their on-the-job supervisors are willing and able to provide such feedback to the supervising teacher.

PORTFOLIOS

Moderate potential. Portfolios can produce longitudinal information, allow students to reflect upon their progress, and give them a voice in assessment. Artifacts could include test scores in classes that covered application topics, papers written on application, journals from internships, reports of projects, etc. The degree to which application is involved in the portfolio design criteria must be departmentally determined.

ASSESSMENT CENTER METHODS

Mixed potential. Simulations are an excellent way to assess application skills, but the necessity for thorough planning and implementation, plus the expense of training or paying already-trained assessors are strong drawbacks of this method. It might be interesting to train senior psychology majors (as part of their capstone experience) to be assessors in simulations conducted in lower-level classes in which psychological principles, theories, and methods are applied (e.g., case studies requiring DSM diagnoses or detection of flaws such as uncontrolled variables in research designs). This would allow faculty to not only involve students in the assessment process, but also provide students with the opportunity to learn and demonstrate a valuable application of psychology (assessment).

CASE AND LONGITUDINAL STUDIES

Mixed potential. Longitudinal assessment studies involve the collection of pre- and post-information and, as such, they can provide evidence of how students change in their ability to apply the discipline over the course of their education. This type of assessment may be most valuable to departments in non-selective institutions whose students enter with minimal skills. These departments may seek to prove that although their students do not graduate with the same high level of skills exhibited by the graduates of more selective school, their students actually make more progress (i.e., more added value) during their undergraduate years than their more high ability counterparts. As with all types of longitudinal design, it is important to realize that pre- and post-changes may be due to factors other than academic programs (e.g., maturity), and that tracking students through the process can be challenging (e.g., students who drop out).

SELF-ASSESSMENT

Strong potential. Both self-assessment methods show promise for assessing application skills in psychology.

STUDENT JOURNALS

Strong potential. Students engaged in internships or service learning projects can be required to journal their experiences by keeping time logs, describing their actual activities, identifying their goals, evaluating whether or not they have accomplished their goals, and illustrating how their goals have been met. A perusal of these journals can give departments an accurate idea of their student's perceptions of their ability to apply the psychological principles and methods they have acquired in the classroom. However, better journal performance is facilitated by explicit directions to reflect application of course content.

SELF-CRITIQUES

Strong potential. When students have criteria that delineate successful performance, they can demonstrate the ability to judge their own skills in applying psychology concepts and principles.

COLLABORATION

Mixed potential. Traditional group projects and research teams show great potential for illustrating application skills; on-line tracking is much more problematic.

RESEARCH TEAMS

Strong potential. One specialized type of group problem-solving is the research team, which requires its members to apply what they have learned in their research methods class in a collaborative setting. The research team can empower students to learn how to apply methods to solve problems without having the full responsibility involved in solo projects. In addition, the number of team members has the potential to enhance the quality of the application just as it has the potential to make completing the project more challenging.

GROUP PROJECTS

Strong potential. Group projects allow faculty to assess their students ability to apply the principles they have acquired in two ways. When students work together to solve problems, they can demonstrate applications in content such as using Kohlberg's stages to determine moral reasoning. Their group work can also illustrate what principles from social psychology can be brought to bear to make the work satisfying (e.g., how to minimize social loafing). Applying these principles is an excellent example of the application of psychological knowledge to both everyday life (e.g., persuading children to do household chores rather than being waited upon by their mothers as if they were members of the royal family) and organizational situations (e.g., getting maximum performance from all members of a committee or work team). Carefully devised rubrics to assess collaboration attitudes and skills (e.g., willingness to volunteer and consensus-building) can be used by both faculty and peers at strategic stages of a project.

ON-LINE ACTIVITIES

Limited potential. Tracking group problem-solving process through online discussion can be a rich source of data for determining the evolution of application skills; however, the disadvantages involved in deconstructing the qualitative materials make this strategy less desirable.

INTERVIEWS & SURVEYS

Moderate potential. Interview methods generally can have application skills as a target but this strategy requires making the assessment of application skills a prominent part of the design.

SATISFACTION SURVEYS

Moderate potential. Satisfaction surveys can be used to determine how well current students or alumni perceive they are learning or learned how to apply psychology. However, the survey must be carefully crafted to reflect an estimate of the student's application skills.

PERFORMANCE REVIEWS

Strong potential. Subsequent work settings provide good contexts in which generalization of skills can be evaluated.

EXIT INTERVIEWS

Moderate potential. Exit interviews can be designed to focus on the aspects of application outlined in this goal.

EXTERNAL EXAMINER INTERVIEWS

Moderate potential. External examiner interviewers usually work from a protocol that should be shaped by the department's interest in the effectiveness of application skills.

FOCUS GROUPS

Strong potential. Focus groups can be used to gather initial data that may zero in on a specific problem. As such, the purpose of the group may be to solve a problem and provide feedback to the department based on the expressed purpose. As such, students can apply principles and concepts in psychology both in the process and product of the focus group.

FOLLOW-UP ALUMNI INTERVIEWS

Limited potential. Calling alumni and identifying examples of successful or not so successful applications of psychology can be a source of data, but the demand characteristics of the situation may produce false positive data. If the purpose is not expressly identified by the researcher, the interview may be suspect on the basis of its potential deception.

ARCHIVAL MEASURES

Mixed potential. In most cases, archival measures cannot provide information about the student's ability to apply psychology. At best, archival records may reveal the intention of course design to address application skills.

TRANSCRIPT ANALYSIS

Limited potential. Transcript analysis might yield the percentage of students engaged in "applied" courses (e.g., internships) as well as the quality of their performance in the class, which could provide a diffuse measure of application skills.

ANALYSIS OF TRANSFER PATTERNS

Not applicable.

SYLLABUS AUDIT

Strong potential. A syllabus audit would be a good first step in determining whether or not faculty are requiring students to engage in assignments that require the application of psychological principles and methods. Where application skills have been identified as a goal by the department, this outcome should be reflected in a reasonable number of syllabi or the department will need to re-examine their curriculum offerings or mission.

DEMOGRAPHIC DATA ANALYSIS

Not applicable.

ALUMNI DATABASE

Limited potential. Determining the percentage of alumni who enter professions that require the application of psychological knowledge and skills would allow a department to get a sense of how successful its curriculum is in preparing students to apply psychology on-the-job. However, the link between job title and application of psychology principles may be fuzzy even for the former student.

LIBRARY STATISTICS USAGE/WEB HITS

Not applicable.

References
Palomba, C. A. & Banta, T. W. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-Bass Publishers.


Goal 5: Values in Psychology
Value empirical evidence, tolerate ambiguity, act ethically and reflect other values that are the underpinnings of psychology as a science. 

5.1

Recognize the necessity for ethical behavior in all aspects of the science and practice of psychology.

5.2

Demonstrate reasonable skepticism and intellectual curiosity by asking questions about causes of behavior.

5.3

Seek and evaluate scientific evidence for psychological claims.

5.4

Tolerate ambiguity and realize that psychological explanations are often complex and tentative.

5.5

Recognize and respect human diversity and understand that psychological explanations may vary across populations and contexts.

5.6

Assess and justify their engagement with respect to civic, social, and global responsibilities.

5.7

Understand the limitations of their psychological knowledge and skills.


ASSESSMENT CATEGORY

ESTIMATE OF METHOD'S POTENTIAL

CLASSROOM/COURSE DATA

Not recommended overall. Although the subtypes demonstrate differential opportunities for assessing values, in general, classroom and course data support other goals more effectively. Direct inquiry into values may be vulnerable to demand characteristics. Inferring values from indirect methods may be prone to interpretive error.

OBJECTIVE TESTS

Not recommended. It may be possible to assess values using this technique but it is unlikely to yield an accurate assessment of the student's true commitment to scientific values.

ESSAY TESTS

Limited potential. Questions that are specifically targeted to inferring and discussing relevant science values may be somewhat helpful, but again the demand characteristics may distort validity.

EMBEDDED QUESTIONS AND ASSIGNMENTS

Limited potential. Departments can embed values checkpoints at various points in required courses, but demand characteristics may influence students to respond in socially desirable ways rather than what they truly believe.

CLASSROOM ASSESSMENT TECHNIQUES

Not recommended. Most classroom techniques concentrate on capturing student understanding of content or appraisal of class effectiveness. Their values may be inferred in the latter purpose but those data tend to be of secondary interest in this application.


INDIVIDUAL PROJECTS/PERFORMANCE ASSESSMENT

Mixed potential overall. Individual projects and other forms of performance assessment do provide some opportunity to assess values based on how students develop their arguments and express what they have learned. Errors (e.g., reliance on personal experience vs. empirical evidence) may provide the basis for a strong inference about which scientific values have not been embraced. Faculty may feel uncomfortable offering feedback based on inferences, no matter how compelling.

WRITTEN PRODUCTS

Strong potential. Written work can assess values in psychology either by addressing explicit values as the focus of the writing or by making errors that reveal the notable absence of an expected value (e.g., when students reports that an experiment "proves" a hypothesis. The presence of designated scientific values in writing projects tends to enhance the overall evaluation of quality of the work since the voice of the paper reflects the values of the community.

ORAL PRESENTATIONS

Strong potential. Oral presentations can also provide significant information about the degree to which students adhere to the values of the psychological community either by the direct values espoused in the presentation or the errors that reveal either a misunderstanding or rejection of those values. Typically faculty do not directly grade presentations based on the values expressed; however, speeches and presentation that more accurately reflect psychology values may exert a positive influence on the grade and feedback.

GRAPHIC TESTS AND DISPLAYS

Limited potential. The abstract nature of values does not lend itself as readily to this type of assessment.

POSTERS

Limited potential. Unless the assigned poster addresses values in an explicit way, faculty may have to infer relevant values from posters designed to address other more concrete concepts. In addition, spontaneous discussion about the poster production can probe student values as one source of data about how the students solved the problem. However, this situation lends itself to strong demand characteristics so students may report the values that will make the instructor happy, not necessarily the true values that motivate their behavior.

STRUCTURAL/SITUATIONAL ASSESSMENTS

Strong potential. Structured problem-solving scenarios can be designed around the scientific values specified in this goal. Students can be asked to resolve some value conflicts in ways that will illustrate whether they have integrated the common values psychologists most typically espouse.


SUMMATIVE PERFORMANCE ASSESSMENT

Mixed potential overall. This category contains some methods that show especially good potential for assessing the values that the student has learned to honor, but other methods are less profitable. The department will need to determine their comfort level with whether and how to assess scientific values.

STANDARDIZED TESTS

Not recommended. There are no commonly used tests of psychological values. There may be more generic scientific value inventories but these have not been implemented.

LOCALLY DEVELOPED TESTS

Not recommended. There are no known inventories on psychological values that have been developed in local contexts.

CAPSTONE EXPERIENCES

Strong potential. The design of the capstone experience can and perhaps should include a component in which the student actively identifies the ways in which their work in the course actively expresses scientific values. Values and ethics may be the content base for the capstone as well, which provides several venues to assess what the student recognizes, understands, and practices among scientific ethics.

INTERNSHIP/PROFESSIONAL APPLICATIONS

Strong potential. Settings in which students can make practical applications of the psychology content they have learned can illustrate the degree to which they have accepted and practiced scientific values. However, not all internship supervisors are likely to have a keenly developed sense of the values expectations. Therefore, a survey or set of criteria may be most useful in framing feedback on the degree to which students illustrate those principles.

PORTFOLIOS

Strong potential. One criterion that can drive selection of work for the student portfolio is the degree to which those products illustrate psychological values. The criteria add legitimacy to faculty comments about the salience of the values in student performance. Including a self-assessment dimension may further foster student's understanding of the critical values.

ASSESSMENT CENTER METHODS

Mixed potential. Not all assessment center methods directly address values. However, it may be possible for students to justify the actions they take in performance assessment situations from the values to which they subscribe. For example, the student might be asked to simulate serving as an editor who needs to choose among three articles, which differ dramatically in the degree to which those papers adhere to the prescribed values. The justification for the selection would reveal the student's values.

CASE AND LONGITUDINAL STUDIES

Strong potential. Qualitative studies of student's changes over time could capture shifts in their adherence to scientific values although traditionally the emphasis in such studies tends to be more focused on the value of the content base itself. Departments would benefit from knowing what elements of the curriculum foster improvements in the practice of scientific values. For example, in which courses are students likely to become comfortable with complexity and ambiguity? Well-framed qualitative investigation would yield such answers.


SELF-ASSESSMENT

Mixed potential overall. Many departments do not pay direct attention to the assessment of psychological values since these may be regarded as abstract or esoteric. In such situations, students will be less well prepared to self-assess. However, to the extent that departments can clarify their expectations about the ways in which they expect student values to change toward greater appreciation of the scientific aspects of psychology, the more student self-assessment can be facilitated.

STUDENT JOURNALS

Mixed potential. In student journals, values can be addressed directly or they may be inferred based on student discussion of related phenomena. Better journal entries will be framed in ways that students can directly discuss their practice of identified values.

SELF-CRITIQUES

Strong potential. Where departments make their values & expectations explicit, students should be able to evaluate the extent to which their own work matches the department's expectations.


COLLABORATION

Limited potential overall. The assessment of the degree to which individuals express scientific values may be challenging to assess in group contexts. Even when the focus of the group activity is directly linked to values, discussion about values may not predict individual behavior. On the other hand, conflict situations may clarify the degree to which students differ in the values that they profess.

RESEARCH TEAMS

Limited potential. Students who work under the direction of a research mentor are likely to receive indirect training on the scientific values that undergird high quality research. When students are challenged to explain why certain actions are required as part of the research process, their understanding and adherence to scientific values can be assessed.

GROUP PROJECTS

Limited potential. Most group projects that transpire in the undergraduate curriculum are unlikely to address scientific values directly. However, some projects could be designed that would allow students to solve problems in such a way that their collective grasp of scientific principles could be demonstrated.

ON-LINE ACTIVITIES

Poor potential. Values may be inferred from group process but the amount of work required make this assessment approach untenable.


INTERVIEWS & SURVEYS

Mixed potential overall. Various approaches in this goal produce differential outcomes in identifying values.

SATISFACTION SURVEYS

Limited potential. Satisfaction surveys do not tend to focus on values related to psychology education. Perhaps some survey items could be crafted to address values, but that might detract from the main purpose.

PERFORMANCE REVIEWS

Limited potential. Inferring other's values from their performance is dicey business. Perhaps it is not best to describe definitively what the values related to psychology education might be.

EXIT INTERVIEWS

Strong potential. Although assessing values will be determined by the concentration of the interview protocol, it is possible to have students conduct some targeted reflection on the values that they have embraced during the course of their education. An additional problem is that the values reported during an interview may not be the values of practice.

EXTERNAL EXAMINER INTERVIEWS

Strong potential. Again, the success in identifying values is dependent on the design of the protocol. It is possible to gain some insight about how values have changed, but personal reports may not correspond to performance realities beyond the interview.

FOCUS GROUPS

Strong potential. A focus group can be convened to address how values change as part of education. However, focus groups tend to have a problem-solving focus apart from values.

FOLLOW-UP ALUMNI INTERVIEWS

Limited potential. Discussing values-related issues with alumni may be a window into their values structure, but the complications of this method, including the problem of deception, make it a less .


ARCHIVAL MEASURES

Not recommended overall. Archival measures generally cannot provide a good gauge of values professed or practiced by psychology students.

TRANSCRIPT ANALYSIS

Not recommended. Although it is possible to infer some values from course selections, there are two many variables that influence course choice for the inferences to be meaningful. In addition, adhering to scientific values cannot be assumed just because science courses have been completed.

ANALYSIS OF TRANSFER PATTERNS

Not recommended. Values cannot be inferred from past coursework.

SYLLABUS AUDIT

Not recommended. Most faculty do not explicitly address the values that a course promotes so an audit is unlikely to produce helpful data about values in psychology education.

DEMOGRAPHIC DATA ANALYSIS

Not recommended. Demographic databases are unlikely to address values in a direct and meaningful manner.

ALUMNI DATABASE

Not recommended. Tracking values explicitly is not a typical feature for the alumni database and doing so could be construed as invasive.

LIBRARY USE/WEB HITS STATISTICS

Not recommended. A checked-out library book or evidence of a web hit does not guarantee that the content has been examined or has created any influence.


Goal 6: Information and Technological Literacy
Demonstrate information competence and the ability to use computers and other technology for many purposes. 

6.1

Demonstrate information competence at each stage in the following process: formulating a researchable topic, choosing relevant and evaluating relevant resources, and reading and accurately summarizing scientific literature. that can be supported by database search strategies

6.2

Use appropriate software to produce understandable reports of the psychological literature, methods, and statistical and qualitative analyses in APA or other appropriate style, including graphic representations of data.

6.3

Use information and technology ethically and responsibly.

6.4

Demonstrate basic computer skills, proper etiquette and security safeguards.


ASSESSMENT CATEGORY

ESTIMATE OF METHOD'S POTENTIAL

CLASSROOM/COURSE DATA

Mixed potential. The only venue in which classroom and course data might reveal information technology expertise would be classes that are heavily mediated. For example, computer labs might be used for on-line testing that would allow some opportunity to gauge student expertise with this method. For the most part, information skills will be better assessed with other strategies that involve activities outside the classroom.

OBJECTIVE TESTS

Limited potential. Knowledge of information and literacy skills could be tested through objective test questions; however, other methods will demonstrate their understanding more directly.

ESSAY TESTS

Poor potential. In class essays would have to focus on student reports of their information retrieval and technological strategies that would produce extremely boring reading.

EMBEDDED QUESTIONS AND ASSIGNMENTS

Strong potential. Departments may dictate specific milestones that should be embedded in required coursework. For example, faculty may specify that exposure to a psychology research database might be embedded in required 101 classes. In smaller contexts, librarians can be enlisted to help conduct information skills training. Later in the curriculum, faculty might identify a courses or set of courses in which they can commit to a particular length of paper with an explicit minimum of high quality scientific sources. The quality of information skills can be inferred from the product; the quality of technological expertise might require more digging or more explicit reporting mechanisms.

CLASSROOM ASSESSMENT TECHNIQUES

Limited potential. This approach may be helpful only in classes specifically focused on the development of information and technology skills.


INDIVIDUAL PROJECTS/PERFORMANCE ASSESSMENT

Mixed potential. Intellectual products provide a stimulus from which faculty can interpret sophistication in information and technological skills. When construction is remote, there may be some risk that the student's work is not an individual production. However, faculty can address this limitation by including a reflective piece that directly addresses the targeted skills.

WRITTEN PRODUCTS

Optimal potential. Written projects provide an ideal context in which to look at research generation, information evaluation, and technology skills.

ORAL PRESENTATIONS

Moderate potential. Individual oral presentations provide some opportunity to evaluate quality of resources; however, these may be challenging to judge in the moment. oral presentations do provide an opportunity to examine power-point or overhead management. In addition, the coherence and development of an oral presentation can reveal research strategies.

GRAPHIC TESTS AND DISPLAYS

Limited potential. Exploring concepts through graphics tends to be an intermediate step in developing research ideas. These may be difficult to assess quality. Such displays may or may not provide an opportunity to assess technology skills,

POSTERS

Moderate potential. The limited space available in most posters may not provide an ideal context in which to evaluate the process of generating research ideas. The poster normally produces highlights so errors or suspect variations may be harder to determine. The execution of the poster will require some technological and aesthetic skills to be successful.

STRUCTURAL/SITUATIONAL ASSESSMENTS

Limited potential. Situational assessments move remote activities into observable territory to facilitate faculty assessment. However, a situational assessment that covers all the outcomes associated with the goal is likely to be fairly intimidating. Performance anxieties may complicate student's ability to perform these complex skills in a situational assessment.


SUMMATIVE PERFORMANCE ASSESSMENT

Mixed potential. Strategies in this category range from zero to maximally helpful.

STANDARDIZED TESTS

Not available. Although there is no standardized approach for measuring research skills, this gap represents an interesting development opportunity.

LOCALLY DEVELOPED TESTS

Limited potential. Some departments have developed methods to assess information skills and research summarizing skills. Assessing these abilities using objective means will be efficient. Asking students to summarize literature will be more challenging and time-consuming but still do-able.

CAPSTONE EXPERIENCES

Strong potential. Assuming that the capstone course has an expressive requirement (e.g., writing or speaking), the capstone course can provide an integrated demonstration opportunity.

INTERNSHIP/PROFESSIONAL APPLICATIONS

Limited potential. The focus of most applied summative experiences will not be focused on the development of the targeted skills.

PORTFOLIOS

Strong potential. Providing explicit criteria that ask students to select ÒworksÓ based on what these reveal about their skills. The evolution of their abilities can be the focus of reflection on why they selected the items they did.

ASSESSMENT CENTER METHODS

Limited potential. Assessment center methods, like situational assessments, move remote activities into observable territory to facilitate faculty assessment. However, a situational assessment that covers all the outcomes associated with the goal is likely to be fairly intimidating. Performance anxieties may complicate student's ability to perform these complex skills.

CASE AND LONGITUDINAL STUDIES

Limited potential. The complexities of tracking these skills over time mitigates against adopting this strategy to monitor their evolution.


SELF-ASSESSMENT

Mixed potential. Developing student's metacognition of their achievement in information and technology skills has not been overtly addressed through self-assessment strategies.

STUDENT JOURNALS

Limited potential. It is unlikely that faculty would choose to invest time reading about student's struggle to learn information and technology skills. However, engaging in student journal writing might provide some keys to faculty about where the particular points of struggle might be.

SELF-CRITIQUES

Strong potential. Where faculty can specify relevant performance criteria, students can provide an estimate of their research conceptualization skills, their sophistication in evaluating information, and their polish in technological execution.


COLLABORATION

Mixed potential. The range of potential in this category ranges from strong to poor.

RESEARCH TEAMS

Strong potential. A research team can develop expertise in research skills through peer involvement. Ironically, research teams may reduce a student's direct involvement in finding resources or producing polished copy. However, the opportunity to brainstorm with peers about developing concepts and executing research strategies makes this an attractive alternative.

GROUP PROJECTS

Moderate potential. Group projects can still involve many of the elements in this target area related to research teams.

ON-LINE ACTIVITIES

Poor potential. Tracking student skill development online will be a complex undertaking unless there is explicit direction for the online traffic to focus on this area of skill development.


INTERVIEWS & SURVEYS

Mixed potential. The assessment of attitudes by the students or other stakeholders may provide some feedback about information/technological competence, but attitudes may not be an accurate indication of true skill attainment.

SATISFACTION SURVEYS

Strong potential. The abilities of students to process information and use technology responsibly can be evaluated by their employers, their graduate advisors, or other stakeholders. External critics may require prompting explicitly to address this skill, but the context in which the critics work provides a reasonable normative comparison.

PERFORMANCE REVIEWS

Strong potential. Graduate school advisors and employers can readily provide comments on the quality of technological preparation for their setting.

EXIT INTERVIEWS

Moderate potential. Students can reflect on the evolution of their information processing and technological execution as part of the interview protocol.

EXTERNAL EXAMINER INTERVIEWS

Limited potential. External examiners can ask probe questions to evaluate student comfort levels about the target skills, but it may be much more challenging to evaluate the quality of their performance from self-report in the absence of concrete evidence. If examiners also review printed materials or tapes of student work, they may be able to make reasonable judgment about student competence.

FOCUS GROUPS

Limited potential. Focus groups most often convene to solve specific problems for a department. Although the topic might be the target of a focus group, it is more likely used for other broader problems.

FOLLOW-UP ALUMNI INTERVIEWS

Not recommended. A follow-up interview on this topic would be hard to execute without inviting demand characteristics that might distort the real skill levels attained.


ARCHIVAL MEASURES

Limited potential.

TRANSCRIPT ANALYSIS

Moderate potential. Transcript analysis can reveal the pattern of courses students may engage in (or avoid) in the development of relevant research skills. The transcript analysis can provide both patterns and some in-class estimates of quality of student performance although the value of these may be limited.

ANALYSIS OF TRANSFER PATTERNS

Limited potential. Departments can benefit by understanding how transfer students may be prepared to engage in research and information activities.

SYLLABUS AUDIT

Moderate potential. Departmental activity examining where research and information skills are taught may be a helpful first step in diagnosing where these skills need to be enhanced.

DEMOGRAPHIC DATA ANALYSIS

Not applicable.

ALUMNI DATABASE

Not applicable.

LIBRARY STATISTICS USAGE/WEB HITS

Not applicable.


Goal 7: Communication Skills
Communicate effectively in a variety of formats.

7.1

Demonstrate effective writing skills in various formats (e.g., essays, correspondence, technical papers, note taking) and for various purposes (e.g., informing, defending, explaining, persuading, arguing, teaching).

7.2

Demonstrate effective oral communication skills in various formats (e.g., group discussion, debate, lecture) and for various purposes (e.g., informing,. defending, explaining, persuading, arguing, teaching).

7.3

Exhibit quantitative literacy.

7.4

Demonstrate effective interpersonal communication skills.

7.5

Exhibit the ability to collaborate effectively.


ASSESSMENT CATEGORY

ESTIMATE OF METHOD'S POTENTIAL

CLASSROOM/COURSE DATA

Moderate to strong potential overall. These methods can be used to assess student's communication skills but only if writing, speaking, and presentation assignments are made part of the coursework.

OBJECTIVE TESTS

Not recommended. Objective tests can be used to assess student's factual knowledge of psychology but have no merit as a metric of student's writing and speaking skills. Poor means of assessing interpersonal skills.

ESSAY TESTS

Strong potential. Essay tests permit careful assessment of student's writing skills.

EMBEDDED QUESTIONS AND ASSIGNMENTS

Moderate potential. This method can be used in any psychology course to assess student's writing skills. Embedding specific departmental assessments to evaluate communication skills in selected courses may be a sound strategy.

CLASSROOM ASSESSMENT TECHNIQUES

Moderate potential. Classroom assessment techniques that focus on measuring student's writing, speaking, interpersonal, and presentation skills may be effective tools for this purpose.


INDIVIDUAL PROJECTS/PERFORMANCE ASSESSMENT

Strong potential overall. All assessment strategies in this category provide direct measures of skills in these areas.

WRITTEN PRODUCTS

Strong potential. Essays, term papers, and laboratory assignments offer perfect opportunities to assess student's conceptual understanding of material; their ability to develop rationale arguments in support of a theory, data, or issues; their understanding of APA style; and language use.

ORAL PRESENTATIONS

Strong potential. Oral reports represent the perfect means of assessing student's public speaking/oral communication abilities.

GRAPHIC TESTS AND DISPLAYS

Strong potential. This method permits assessment of student's abilities to communicate information, particularly numerical data, in a visual medium.

POSTERS

Strong potential. This method permits simultaneous assessment of student's writing skills, graphic display skills, and oral communication skills.

STRUCTURAL/SITUATIONAL ASSESSMENTS

Strong potential. Placing students in situations that require them to role play, participate in mock interviews, and so on, may be an effective means of assessing their ability to think on their feet, speak extemporaneously, and interact with each other.


SUMMATIVE PERFORMANCE ASSESSMENT

Limited to moderate potential overall. Recommendations vary in this category from strong potential to not recommended.

STANDARDIZED TESTS

None available. There are no existing national normed tests to address communication skills in psychology in summative performance.

LOCALLY DEVELOPED TESTS

Limited to Strong potential. Locally developed essay tests permit assessment of student's writing skills; objective tests do not.

CAPSTONE EXPERIENCES

Moderate to Strong potential. Capstone courses that include a writing or speaking component represent excellent opportunities to assess senior student's communication abilities. Group activities may be useful in assessing student's interpersonal skills.

INTERNSHIP/PROFESSIONAL APPLICATIONS

Limited potential. Unless the internship/professional application involves writing or speaking components that are directly assessed, this method holds little promise for assessing student's communication skills. However, this method may be useful for assessing student's abilities to collaborate with others in a real-life setting, thus providing information on their interpersonal skills.

PORTFOLIOS

Strong potential. portfolios entail collections of written work that has been created over time and thus represent an effective means of assessing the development of student writing skills.

ASSESSMENT CENTER METHODS

Limited potential. In-basket strategies and other assessment methods need to build in explicit communication tasks to qualify for consideration.

CASE AND LONGITUDINAL STUDIES

Mixed potential Longitudinal tracking of student's communication abilities over time can be a useful source of information, however, the complexities of this approach (e.g., storage, feedback intensiveness) discourages its use.


SELF-ASSESSMENT

Moderate to Strong potential overall. Self-assessment can be used effectively in almost any psychology course. While self-assessment strategies permit insight into student's academic experiences, they vary in value for assessing student's communication abilities. In many cases, faculty construct self-assessment documents casually and this practice may limit opportunity to examine student's polished communication skills.

STUDENT JOURNALS

Moderate to Strong potential. The usual purpose of journal assignments is to encourage personal expression and insight rather than as a vehicle for assessing communication skills; however, adding specific communication criteria to journal directions can facilitate assessment in this area.

SELF-CRITIQUES

Moderate to Strong potential. Students can provide judgments of their own communication strengths and weaknesses although personal bias may limit the accuracy of their judgments. Clearly established communication criteria and developmental practice in using the criteria will facilitate the best results.


COLLABORATION

Limited to Moderate potential overall. Group projects offer students a chance to collaborate, but do not guarantee the chance to assess individual student communication skills. However, these methods do hold potential for assessing student's abilities to work in groups.

RESEARCH TEAMS

Limited potential. The primary objective of most research teams concentrates on the product (i.e., empirical research, data collection). Although the development of communication skills can be assessed, such comprehensive attention to student contributions is not typical.

GROUP PROJECTS

Moderate potential. Group projects permit attention to both product and process. However, in practice most faculty pay greater attention to the quality of the product than the process.

ON-LINE ACTIVITIES

Not recommended Monitoring electronic communication archives to assess communication skills seems indirect and time-consuming.


INTERVIEWS & SURVEYS

Mixed potential overall. Evaluating communication through interview or survey presents mixed potential, influenced substantially by selected format. Objective/Likert type surveys permit almost no opportunity to measure student's communication skills; open-ended survey questions are limited in assessing student's written communication skills because they generally require very short "quick" responses.

SATISFACTION SURVEYS

Limited potential. These types of surveys only marginally examine written communication skills.

PERFORMANCE REVIEWS

Moderate potential. External assessors can provide feedback on quality of communication skills that will reflect their specific communication concerns (e.g., employers can provide feedback on the effectiveness of communication abilities within the worksite).

EXIT INTERVIEWS

Strong potential. Although this type of interview can be time-consuming, it may permit effective assessment of senior student's oral communication and interpersonal skills.

EXTERNAL EXAMINER INTERVIEWS

Strong potential. External examiners can also assess oral communication and interpersonal skills with the potential additional advantage of greater objectivity.

FOCUS GROUPS

Limited to Moderate potential. This method may permit assessment of student's social interaction abilities and skills but is generally of limited use because of time and logistical considerations.

FOLLOW-UP ALUMNI INTERVIEWS

Limited to Moderate potential. This approach can be problematic since communication skills would be indirectly assessed in this approach.


ARCHIVAL MEASURES

Not recommended overall. These methods reveal little useful information about student's communication abilities.

TRANSCRIPT ANALYSIS

Not recommended. A transcript can provide information on formal coursework that should contribute to communication skill development but does not address individual achievement.

ANALYSIS OF TRANSFER PATTERNS

Not recommended. Transfer patterns may alert faculty to gaps in student's communication course preparation, but do not directly address skill levels.

SYLLABUS AUDIT

Limited potential. Departments can determine the extent to which specific communication skills are built into the curriculum but quality of experience is not directly observable.

DEMOGRAPHIC DATA ANALYSIS

Not recommended. No pertinent data can be gained through this process.

ALUMNI DATABASE

Not recommended. No pertinent data can be gained through this process.

LIBRARY USE/WEB HITS STATISTICS

Not recommended. An analysis of this type can reflect student's formal interest in communication related books or websites, but does not provide direct information on skill attainment.


Goal 8: Sociocultural and International Awareness
Recognize, understand and respect the complexity of sociocultural and international diversity.

8.1

Interact effectively and sensitively with people from diverse backgrounds and cultural perspectives.

8.2

Examine the sociocultural and international contexts that influence individual differences.

8.3

Explain how individual differences influence beliefs, values, and interactions with others and vice versa.

8.4

Understand how privilege, power, and oppression may affect prejudice, discrimination, and inequity

8.5

Recognize prejudicial attitudes and discriminatory behaviors that might exist in themselves and others.


ASSESSMENT CATEGORY

ESTIMATE OF METHOD'S POTENTIAL

CLASSROOM/COURSE DATA

Overall Mixed potential. In class strategies vary in their effectiveness in gauging learning in this diversity-related goal.

OBJECTIVE TESTS

Mixed potential. Objective tests can provide in-class assessment of facts and principles, but other approaches are likely to produce greater gains.

ESSAY TESTS

Mixed potential. Essay questions can provide a great vehicle for evaluating student knowledge of diversity-oriented facts and also for allowing students the opportunity to reflect on key ideas in coherent essays. The drawback is that the topics may produce strong demand characteristics that may produce discrepant predictions from real behavior.

EMBEDDED QUESTIONS AND ASSIGNMENTS

Strong potential. Many departments have opted to designate specific courses as "diversity oriented" and entrust these courses to provide the learning experiences and associated assessments to accomplish the outcomes related to this goal. To the extent that the department collaborates on the design of the outcomes or learning experiences, this approach can be profitable. Sharing outcomes across a small number of courses may help this goal become more manageable.

CLASSROOM ASSESSMENT TECHNIQUES

Moderate potential. Private reactions at critical moments in a course may be a powerful learning vehicle for students. When teachers collect those observations, anonymous reactions may allow students to be more candid. From that collection, the faculty member may get a meaningful index about how the students are relating to topics as a whole but are unlikely to provide individual feedback to students in this very personal domain.


INDIVIDUAL PROJECTS/PERFORMANCE ASSESSMENT

Overall Strong potential. Because insights related to this goal are intensely personal, individual performances may be one of the most effective routes for assessing achievement in this area.

WRITTEN PRODUCTS

Strong potential. Whether the projects focus on cultural/ethnic differences or merely emphasize this area as part of a larger effort, written projects provide an opportunity to explore content as well as personal reactions. Better assignments will be driven by a rubric in which performance related to this goal can be made specific.

ORAL PRESENTATIONS

Strong potential. The advantages are similar to written products with the additional advantage that student learning can be probed by the audience members.

GRAPHIC TESTS AND DISPLAYS

Moderate potential. Concept maps can produce some preliminary ideas related to this goal, but are unlikely to serve as final products.

POSTERS

Strong potential. Posters can provide a great way for many students to demonstrate their learning. Rubrics can provide clear expectations about criteria for achievement.

STRUCTURAL/SITUATIONAL ASSESSMENTS

Moderate potential. The demand characteristics of an assessment in diversity related learning runs some risk of producing artificial responses that may not be a true representation of student's attitudes. However, a situational assessment can be a reasonable measure of content knowledge in this area.


SUMMATIVE PERFORMANCE ASSESSMENT

Mixed potential. Although summative approaches can be used to address this area, none of the methods in this category produce optimal strategies.

STANDARDIZED TESTS

None available. At the present time there is no standardized assessment instrument to address diversity knowledge and attitudes.

LOCALLY DEVELOPED TESTS

Poor potential. It is hard to imagine that departments would concentrate on sociocultural and international dimensions in a locally developed objective test. Dedicating even just a few items on a broader test is unlikely to produce a helpful picture of student development in this arena.

CAPSTONE EXPERIENCES

Moderate potential. A capstone course can be dedicated to cross-cultural, ethnic, or international concerns so performance in a capstone can be assessed for this dimension.

INTERNSHIP/PROFESSIONAL APPLICATIONS

Limited potential. Internships and field experiences can provide students with an opportunity to engage with populations that are divergent from their own, providing a rich venue in which to develop diversity-related skills.

PORTFOLIOS

Moderate potential. A diversity component can be a required feature in a portfolio. This requirement will only be workable in programs in which learning experiences produce meaningful opportunities to demonstrate diversity-related content and skills.

ASSESSMENT CENTER METHODS

Moderate potential. In-basket strategies or other assessment center methods can provide diversity-oriented learning opportunities; however, the demand characteristics of this approach may render performances that are more staged than genuine.

CASE AND LONGITUDINAL STUDIES

Moderate potential. The expense and time requirements of case and longitudinal studies of how students achieve their learning goals militate against the use of this method to address diversity concerns. In addition, changes in attitude and knowledge may not be a direct function of the curriculum so the findings of such studies may be misleading. However, examining attitude and content knowledge in this area would produce valuable information about the role of curriculum in facilitating this change.


SELF-ASSESSMENT

Mixed potential. Self-awareness can be a critical component of successful educational experiences in diversity-related content and skill development.

STUDENT JOURNALS

Strong potential. When students have the opportunity to learn about diversity-related issues, a journal may be a strong, if not optimal, way to allow personal exploration of controversial matters.

SELF-CRITIQUES

Moderate potential. Self-review can be a meaningful form of assessment in this arena when faculty develop clear and meaningful criteria for performance.


COLLABORATION

Overall favorable potential. Learning about differences by actual participation in groups may be an ideal way to develop diversity-related skills assuming that students get objective feedback based on their performances.

RESEARCH TEAMS

Strong potential. Research teams devoted to solving problems related to intercultural and international issues can provide an excellent vehicle for students to develop expertise in the content of this area.

GROUP PROJECTS

Strong potential. Group projects can provide direct experience for students who are learning to work with others who may be different from themselves. The nature of the project may also lend itself to enhanced learning of the content in this area.

ON-LINE ACTIVITIES

Moderate potential. Online archives provide concrete evidence of social interaction across participants, which can be very useful in deconstructing conflict or otherwise understanding what makes diverse people work more effectively. Unfortunately, this method of assessment is quite labor intensive and may feel deceptive unless the students understand that their communications will be monitored for learning purposes.


INTERVIEWS & SURVEYS

Mixed potential. Interviews and surveys may not be the most strategic way to assess genuine diversity-related gains due to demand characteristics of the situation.

SATISFACTION SURVEYS

Limited potential. Surveying attitudes about diversity-related skills is likely to be perilous due to the strong demand characteristics of such a survey. However, it may be reasonable to include satisfaction items with the content and skills that encourage positive relating to diverse others.

PERFORMANCE REVIEWS

Limited potential. Stakeholders may not be able to describe sociocultural values and behaviors beyond the most basic of behaviors.

EXIT INTERVIEWS

Moderate potential. Although it may be reasonable to incorporate questions as part of an exit interview regarding how well the curriculum has prepared students to deal with diversity, the demand characteristics complicate the interpretation of the results.

EXTERNAL EXAMINER INTERVIEWS

Limited potential. Students are likely to report attitudes that are consistent with creating a favorable impression. However, an external examiner's neutrality may provide a greater opportunity to assess realistic attitudes than methods that rely on program faculty.

FOCUS GROUPS

Strong potential. When a focus group is convened for the specific purpose of solving problems related to diversity, students can experience a rich learning opportunity that also provides markers of their growth. For example, a focus group might be designed to address ways to increase exposure to different peoples and cultures, which would allow for the assessment of attitudes and content.

FOLLOW-UP ALUMNI INTERVIEWS

Poor potential. A follow-up with alumni that assesses attitudes and skills without necessarily disclosing that purpose is fraught with problems. Although a well-designed protocol may produce realistic responses, the ethical complications of such an approach discourage its use.


ARCHIVAL MEASURES



TRANSCRIPT ANALYSIS

Limited potential. A transcript analysis can only provide gross information about the presence or absence of courses devoted to diversity-related themes.

ANALYSIS OF TRANSFER PATTERNS

Limited potential. Transcripts of transfers can provide some information about the degree to which new students may be bringing in diversity-related coursework. Understanding whether transfer students will have already engaged in such coursework can facilitate more targeted planning for the new institution.

SYLLABUS AUDIT

Moderate potential. Departments would benefit from an audit of existing syllabi to determine the extent to which this outcome is a routine part of the curriculum.

DEMOGRAPHIC DATA ANALYSIS

Moderate potential. Analysis of the demographics of students may be helpful in determining what kinds of experiences with other cultures or ethnic traditions would be useful in building awareness and related skills.

ALUMNI DATABASE

Limited potential. Tracking the characteristics of the database may be useful in terms of linking the student body with the alumni. It may be useful to target alums from specific traditions to assist in diversity training.

LIBRARY STATISTICS USAGE/WEB HITS

Limited potential. Faculty can learn whether international, cultural, and ethnic resource books see much action from the students, but user rates don't reliably indicate what has been learned by the students. More direct strategies are preferable.


Goal 9: Personal Development
Develop insight into their own and other's behavior and mental processes and apply effective strategies for self-management and self-improvement.

9.1

Reflect on their experiences and find meaning in them.

9.2

Apply psychological principles to promote personal development.

9.3

Enact self-management strategies that maximize healthy outcomes.

9.4

Display high standards of personal integrity with others.


ASSESSMENT CATEGORY

ESTIMATE OF METHOD'S POTENTIAL

CLASSROOM/COURSE DATA

Mixed potential. Departments will naturally vary in relation to the importance of the personal development goal; however, the primary emphasis in most classroom activity and course assignments will be mastery of the appropriate content. Unless the courses itself has a particular or purposeful personal development focus or feature, it is unlikely much assessment of related outcomes will occur in courses.

OBJECTIVE TESTS

Limited potential. Personality inventories, such as the Myers-Briggs SIGI-PLUS, or assorted learning inventories can be used to develop some developmental strategies in classes that have a personal development emphasis (e.g., college success courses). The Multimedia Integrity Test (MIT) also provides an inventory related to integrity. However, a specific inventory has not been developed for this category, would be very expensive to develop, and would have narrow use in a typical psychology curriculum.

ESSAY TESTS

Strong potential. Although the quality of feedback will depend on the clarity of instructions, well designed essay questions can be used to prompt personal reflection.

EMBEDDED QUESTIONS AND ASSIGNMENTS

Strong potential. Self-assessment strategies that are routinely embedded in course assignments will help develop strengths regarding self-evaluation, goal-setting, and other aspects of meta-cognition and self-regulation.

CLASSROOM ASSESSMENT TECHNIQUES

Strong potential. Classroom strategies that require students to construct meaning and make active connections to prior learning have the potential to build personal development skills if the items are designed to produce that outcome.


INDIVIDUAL PROJECTS/PERFORMANCE ASSESSMENT

Mixed potential. Personal development goals may seen as a relevant to only a limited range of courses of educational experiences. However, individual projects provide good samples for personal reflection.

WRITTEN PRODUCTS

Strong potential. Journals, essays with a personal focus, letters, or other means of personal expression are better adapted to this goal than formats designed for scientific communication. Such work will necessarily have a strong emphasis on self-assessment.

ORAL PRESENTATIONS

Limited potential. The emphasis on oral presentation training in most programs is geared toward professional speeches on research or concepts. Self-assessment strategies deployed to evaluate the success or weaknesses of presentations can enhance student understanding of self-development, but it is unlikely that speeches would be devoted to student exploration of personal development insights, which would likely be viewed as a narcissistic, boring, or time-wasting enterprise.

GRAPHIC TESTS AND DISPLAYS

Limited potential. Due to the personal nature of the objectives in this category, graphic representation would have little appeal for broad audiences. Metaphor work may provide a reasonable expression about how students experience personal development challenges.

POSTERS

Limited potential. Due to the personal nature of the objectives in this category, it is hard to imagine that a poster would have appeal for broad audiences.

STRUCTURAL/SITUATIONAL ASSESSMENTS

Strong potential. Authentic problem-solving situations can be structured to promote student learning in self-development. Design of performance assessments need to incorporate careful rubric development to foster on-target developmental feedback related to this skill development area.


SUMMATIVE PERFORMANCE ASSESSMENT

Mixed potential. Assessment designed to capture penultimate achievement can include personal development dimensions although not all formats do so with equal success.

STANDARDIZED TESTS

Not available/recommended. There is no national test of personal development skills in psychology per se. Should such an exam be developed in this area, it would provide some interesting opportunities for benchmarking how students develop metacognitively. Some aspects of this goal can be addressed by values inventories.

LOCALLY DEVELOPED TESTS

Not available/recommended. There are no current exemplars of personal development exams developed locally.

CAPSTONE EXPERIENCES

Strong potential. Capstone courses offer an ideal venue in which students can review how their personal development has unfolded during the undergraduate education. The review of self-regulation competence should also assist in the preparation of students for job interviews or graduate school application.

INTERNSHIP/ PROFESSIONAL APPLICATIONS

Strong potential. Applied settings offer an ideal context in which to practice self-monitoring, application of criteria, strategies for self-regulation, etc. Feedback protocol will need to focus on these reflective aspects of performance to elicit appropriate measurement.

PORTFOLIOS

Strong potential. Assembled works and the processes used to produce them are the heart of the portfolio process. A reflective protocol (e.g., asking students to explain how a selected work provides evidence for a claim about personal development) facilitates student and faculty commentary on the personal development patterns exhibited by the students over time.

ASSESSMENT CENTER METHODS

Strong potential. Authentic problem-solving situations can be structured to simulate the choices that students will make related to time-management (e.g., in-basket strategies) or ethical behavior (e.g., moral dilemma discussions). Feedback can be structured about personal development based on the design of the performance. assessment.

CASE AND LONGITUDINAL STUDIES

Strong potential. The qualitative aspects of longitudinal studies lend themselves more readily to assessment of personal development. Some values inventories may provide assistance to track the course of personal development over time.


SELF-ASSESSMENT

Optimal. The personal nature of the goals and objectives in this category lend themselves in an ideal way to the promotion of self-assessment skills. Programs that emphasize this approach strive to build self-assessment as a development skill that begins with students making general judgments about the quality of their work and progresses to higher level demands, including creating and applying performance criteria to evaluate what students have accomplished.

STUDENT JOURNALS

Strong potential. With appropriately designed prompts, student journalscan address any of the objectives in this category. Although this strategy is a good match, it is also very vulnerable to the personal biases and filters that may not make self-reports a valid interpretation. Faculty report best success when rubrics specify the criteria by which students will be effective in personal development goals.

SELF-CRITIQUES

Strong potential. Holding students accountable for judging their own performances promotes metacognitive development. Practice with self-assessment may promote habitual reflection and self-criticism. Personal biases (e.g., self-esteem issues, grandiosity) may make interpretations less reliable and valid.


COLLABORATION

Mixed potential. Personal development outcomes express individual achievement; however, group contexts can provide opportunities for development of an effective group persona. Emphasis on production or solution may obscure effective evaluation of personal achievement as a group member.

RESEARCH TEAMS

Strong potential. Participating in groups that must be sustained over time provide a particularly helpful opportunity to monitor personal development related to interaction skills. This outcome will be enhanced where students receive explicit feedback on the quality of their contributions as well as the quality of their product or solution.

GROUP PROJECTS

Limited potential. Short-term projects offer some opportunity to gain insight into interpersonal skills. However, most faculty may not feel comfortable or have access to the mechanics of the group to provide developmental feedback on group skills.

ON-LINE ACTIVITIES

Limited potential. Online interactive exchanges provide solid evidence of skills or deficits in group communication. However, faculty may feel offering uncomfortable about offering developmental feedback on this skill unless they make it an explicit feature of this learning opportunity and are prepared to invest the time that it would take to decode the online exchanges.


INTERVIEWS & SURVEYS

Mixed potential. The purpose of these strategies tends to be straightforward program evaluation as opposed to methods that can more directly influence student learning. Progress in personal development or success in its dimensions can be designed as a central feature of a specific process. Quality of results will be dependent on the design features of the survey instrument or interview protocol.

SATISFACTION SURVEYS

Mixed potential. Employers, parents, and other stakeholders in a student's education may be able to comment accurately about many aspects of personal development (e.g., ethics, time management, insightfulness); however, the demand characteristics and selection challenges may threaten the validity of the results.

PERFORMANCE REVIEWS

Mixed potential. Stakeholders may not be able to link current behaviors to personal development status without substantial training.

EXIT INTERVIEWS

Strong potential. Although the timing of the interview may encourage a positive response bias, exit interviews can be a good opportunity to incorporate student review of achievements in personal development.

EXTERNAL EXAMINER INTERVIEWS

Strong potential. A protocol that directly addresses personal development can provide an opportunity for the interviewer to probe suspect responses from the student potentially to improve the validity of the student's responses. Personal development objectives need to be made explicit in the interview protocol to produce consistent departmental data on how they facilitate these outcomes.

FOCUS GROUPS

Limited potential. Focus groups are unlikely to address personal development concerns unless that is the express objective of the department's convening the group. The group format may be sufficiently inhibiting in relation to self-disclosure that this strategy is not recommended.

FOLLOW-UP ALUMNI INTERVIEWS

Limited potential. Phone interviews designed to collect data about the personal development accomplishment of the alumni are likely to be suspect. The rationale for contacting the alumni may be difficult to address without creating a demand situation in which the alum feels some pressure to respond positively.


ARCHIVAL MEASURES

Not recommended. Archival measures would typically not allow for direct measurement of personal development objectives.

TRANSCRIPT ANALYSIS

Limited potential. A transcript analysis would only reveal student grades in courses that might be designated more for personal development. Such analyses would tell us little about the actual skills and values students would demonstrate.

ANALYSIS OF TRANSFER PATTERNS

Not appropriate. The question of transfer patterns does not have a direct bearing on personal development.

SYLLABUS AUDIT

Limited potential. To the extent that faculty completely address the goals that they may have for student development, a syllabus audit can provide good information about the extent to which this is a highly valued outcome for the department.

DEMOGRAPHIC DATA ANALYSIS

Not appropriate. Demographics do not directly address dimensions of personal development.

ALUMNI DATABASE

Limited potential. Depending on the scope of data maintained in the database, some personal development characteristics could be gleaned but it is likely to be labor intensive and inferential under the best of circumstances.

LIBRARY STATISTICS USAGE/WEB HITS

Limited potential. Faculty could track check-out patterns for books that are devoted to personal development or web sites, but these data would be quite remote from giving direct information about student development patterns.

 

Goal 10: Career Planning and Development
Pursue realistic ideas about how to implement their psychological knowledge, skills and values in occupational pursuits in a variety of settings.

10.1

Apply knowledge of psychology (e.g., decision strategies, life span processes, psychological assessment, types of psychological careers) to formulating career choices.

10.2

Identify the types of academic experience and performance in psychology and the liberal arts that will facilitate entry into the work force, post-baccalaureate education, or both.

10.3

Describe preferred career paths based on accurate self-assessment of abilities, achievement, motivation, and work habits.

10.4

Identify and develop skills and experiences relevant to achieving selected career goals.

10.5

Demonstrate an understanding of the importance of lifelong learning and personal flexibility to sustain personal and professional development as the nature of work evolves.


ASSESSMENT CATEGORY

ESTIMATE OF METHOD'S POTENTIAL

CLASSROOM/COURSE DATA

Departments that have Careers in Psychology course will find these methods more useful than departments that do not. Even in such courses, however, this method has limited potential because it does not typically lend itself to assessing individual student's career plans and development.

OBJECTIVE TESTS

Limited potential. Objective tests can be used to assess student's factual knowledge of career/educational options and related information, but they do not permit an in-depth assessment of individual student's post-baccalaureate plans.

ESSAY TESTS

Mixed potential. Essay tests permit personalized assessment of student's post-baccalaureate plans. Nonetheless, other methods can more effectively assess this information.

EMBEDDED QUESTIONS AND ASSIGNMENTS

Limited potential. This method can be used in Careers in Psychology courses, but other methods can more directly assess this information. This method is less practical in other courses.

CLASSROOM ASSESSMENT TECHNIQUES

Mixed potential. This method can be used in any course to assess student's understanding of career planning and development; faculty may object to using class time for this purpose.


INDIVIDUAL PROJECTS/PERFORMANCE ASSESSMENT

Strong potential overall. These methods can be used quite effectively in Careers in Psychology, Psychology of Adjustment, some developmental psychology course, and some capstone courses. It offers a number of useful means by which to assess individual student's post-baccalaureate career planning. Uses outside these courses are limited.

WRITTEN PRODUCTS

Strong potential. Students can prepare résumés, plot out courses they need to take prior to graduation, calculate their likely and highest possible grade-point averages, conduct informational interviews, and report on selected graduate schools.

ORAL PRESENTATIONS

Strong potential. Students can give oral reports on careers relevant to psychology.

GRAPHIC TESTS AND DISPLAYS

Limited potential. Concept maps can illustrate some key features of career planning, but other methods show more promise.

POSTERS

Strong potential. Students can prepare "posters" that detail key aspects of one or more careers or graduate schools.

STRUCTURAL/SITUATIONAL ASSESSMENTS

Strong potential. Some Career Services offices conduct mock job interviews with students; these can videotaped to facilitate feedback.


SUMMATIVE PERFORMANCE ASSESSMENT

Mixed potential overall. Approaches in this category vary from "strong" to "not recommended."

STANDARDIZED TESTS

Not recommended. No national instrument to address career knowledge and planning exists.

LOCALLY DEVELOPED TESTS

Not recommended. It is unlikely a department would devote time to this kind of assessment project.

CAPSTONE EXPERIENCES

Mixed potential. Some capstone courses might include a project wherein students research and write about their career goals and/or graduate school plans.

INTERNSHIP/PROFESSIONAL APPLICATIONS

Strong potential. These experiences can help students clarify their career goals and develop knowledge, skills, and values that will facilitate their entry into the workforce.

PORTFOLIOS

Limited potential. One objective of the construction of portfolios is for assistance in career planning but it is unlikely that a portfolio would be devoted solely to career pursuits.

ASSESSMENT CENTER METHODS

Strong potential. In-basket simulations can mirror closely the kinds of skills students might need to demonstrate in different career paths. As such, these methods can provide important data to students and their programs.

CASE AND LONGITUDINAL STUDIES

Mixed potential. Following the career goals and paths of graduates may provide a rich source of data but changing historical contexts may limit the value of investing faculty time in this type of assessment.


SELF-ASSESSMENT

Strong potential overall. These methods can be used effectively in Careers in Psychology, Psychology of Adjustment, some developmental psychology courses, and some capstone courses. They allow for in-depth assessment of individual student's post-baccalaureate career plans.

STUDENT JOURNALS

Strong potential. Students can reflect on their career and educational goals, ascertain what additional information they need, obtain this information, incorporate it into their planning, refine their goals, and report their progress over the term.

SELF-CRITIQUES

Strong potential. Although self-critiques may lack objectivity, systematic use of this practice can enhance student's ability to develop insight into the quality of their own performances.

COLLABORATION

Limited potential overall. Because students must develop their own goals, group projects are not particularly helpful.

RESEARCH TEAMS

Limited potential. Research teams can convene to develop expertise on career research but such activities will be limited to correlational and survey methods.

GROUP PROJECTS

Good potential. Student groups can complete projects related to career goals to enhance their understanding and promote their own career planning.

ON-LINE ACTIVITIES

Limited potential. Monitoring web-based activities to assess career awareness seems misdirected.


INTERVIEWS & SURVEYS

Strong potential overall. These methods can be used in Careers in Psychology, Psychology of Adjustment, some developmental psychology courses, and some capstone courses. These methods can also be used to gather data in other classes or in non-classroom settings, but faculty may object to using class time for this purpose.

SATISFACTION SURVEYS

Strong potential. There is a problem of where to administer surveys if not given in career-relevant courses.

PERFORMANCE REVIEWS

Not recommended. Asking employers to comment on student's career preparation may seem invasive.

EXIT INTERVIEWS

Strong potential. These can be used to assess student's post-baccalaureate plans and their views of departmental resources for and support of career planning and development. These can be time-consuming.

EXTERNAL EXAMINER INTERVIEWS

Limited potential. Other methods are more effective. External examiner time is usually invested in verifying other skills.

FOCUS GROUPS

Limited potential. Focus groups represent a relatively inefficient way to gather information about student's views of departmental resources for and support of career planning and development;some concerns about student's objectivity in a group setting.

FOLLOW-UP ALUMNI INTERVIEWS

Strong potential. Although time-consuming, this method can provide useful information regarding the department's effectiveness in preparing students for entry-level careers or graduate study.


ARCHIVAL MEASURES

Limited potential overall. Options in this category range from inappropriate to highly desirable.

TRANSCRIPT ANALYSIS

Not recommended. It is unlikely that this information would be helpful in assessing student's career goals and development.

ANALYSIS OF TRANSFER PATTERNS

Not recommended. Career concerns cannot be interpreted from transfer patterns.

SYLLABUS AUDIT

Limited potential. Departments might benefit from examining whether there are specific career-related goals present in the curriculum, especially in advanced coursework.

DEMOGRAPHIC DATA ANALYSIS

Not recommended. There is no direct value from this kind of analysis.

ALUMNI DATABASE

Strong potential. Tracking students regarding employment and graduate degrees can provide useful (aggregated) data regarding the variety and level of jobs that majors typically obtain and the proportion of majors who attend graduate/professional school. These data can be tracked over time in relation to career-relevant changes in the curriculum, development of new advising resources, etc.

LIBRARY USE/WEB HITS STATISTICS

Not recommended. A checked-out library book or evidence of a web hit does not guarantee that the content has been examined or has created any influence.