Step 1: Describe where-when-how
To determine where/when to conduct assessment, the first thing to consider is the minimum assessment requirement, so that you conduct assessment in a manner that fulfills the requirement:
NAU requires that minimally, all broad learning outcomes are assessed using direct methods of assessment toward or at the end of the student’s program of study at least once between program reviews.
Using the capstone course or “experience”
Adapted from Western Washington University’s Tools & Techniques for Program Improvement: Handbook for Program Review & Assessment of Student Learning (2006) and the University of Wisconsin, Madison, Outcomes Assessment Manual (2000).
The most promising course to review for use in assessment is the capstone course for undergraduate programs, or, if you are assessing a graduate program, we would recommend starting by reviewing the comprehensive exams and/or thesis/dissertation.
NAU’s undergraduate capstone requirement, and comprehensive exams or theses/dissertations in graduate programs, are designed to provide a culminating experience within the degree, wherein students inquire, synthesize, apply, or work in ways that prepare them for a productive future; or demonstrate the breadth and depth of their learning through comps and theses. Since the capstone is a culminating experience that is meant to ensure students demonstrate the ability to synthesize and integrate knowledge and experiences of learning throughout their program, it’s very likely that most, if not all, learning outcomes for the degree program are incorporated into the capstone course. But, you can only use the capstone as a key place to collect outcome measures if:
- the course and its assignments are truly representative of requirements for the degree;
- the course curriculum and assignment evaluation (or products) are consistent across sections or instructors or comprehensive exam/thesis/dissertation committees; and
- students understand the value and importance of the capstone course, senior assignment, or comprehensive exam/thesis/dissertation and take this requirement seriously—meaning their performance is most likely reflects their best effort.
More about using the capstone for assessment Accordion Closed
*adapted from Ball State University Assessment Workbook (2012)
Capstone courses: Capstone courses provide an opportunity to measure student learning, because this is where students are most likely to exhibit their cumulative understanding and competence in the discipline. One of the purposes of capstone courses is to provide an opportunity for students to “put it together,” which typically requires students to integrate the knowledge, skills and abilities found in the program’s learning outcomes.
Culminating assignments offer students the opportunity to put together the knowledge and skills they have acquired in the major, provide a final common experience for majors, and offer faculty a way to assess student achievement across a number of discipline-specific areas. Culminating assignments are generally designed for seniors in a major or field to complete in the last semester before graduation. Their purpose is to integrate knowledge, concepts and skills that students are expected to have acquired in the program during the course of their study. This is obviously a curricular structure as well as an assessment technique and may consist of a single culminating course (a “capstone” course) or a small group of courses designed to measure competencies of students who are completing the program. A senior assignment is a final culminating project for graduating seniors such as a performance portfolio or a thesis that has the same integrative purpose as the capstone course.
Advantages of using the capstone course or a culminating experience
- When capstone courses or projects are required, they can provide an ideal data collection opportunity because seniors are accessible.
- Assessments can provide an opportunity to motivate students through the curriculum. Also they can provide quality data that permit meaningful reflection on the program.
- Seniors are well into the curriculum and can reflect on their learning experience and the curriculum.
- These assessment methods provide seniors with an opportunity to provide meaningful feedback when they believe that their opinions are respected and valued.
- Students get feedback on their accomplishments, and student responsibility is encouraged.
- They can be used for both student evaluation (assess seniors’ overall ability and knowledge gained from the program) and program evaluation (annual, continuous evaluation of curriculum from student feedback).
- They support program coherence.
- They provide an opportunity to create local assessment instruments that can be used in conjunction with other methods, such as surveys and standardized tests.
- Many faculty members are engaged in planning the topics and the design of the capstone experience.
- This assessment allows flexible course content (i.e., adaptable to different courses).
Disadvantages
- Student performance may be impaired due to “high stakes” of the project. Successfully completing the capstone course may be a requirement for graduation, which may generate some anxiety for both faculty and students.
- A faculty member may develop the idea that the capstone course or project should only involve him or her.
Considerations
- Ensure that the course assignments or projects accurately represent the major or program requirements.
- Use checkpoints to prevent difficulties, especially towards the end, which may affect a student’s graduation.
- Maintain the curriculum and evaluation of assignments across all sections.
- Ensure that students understand and value the importance of the capstone experience and take it seriously.
- Secure administrative support before implementing a capstone experience since there are usually high costs associated with it because of the small class size required to maximize the faculty-student interaction.
- Design capstone course or project to assess curriculum goals and outcomes.
Keep in mind:
– adapted from the University of Massachusetts, Amherst “Program-Based Review & Assessment Tools & Techniques for Program Improvement,” (April 2017).
Assessment data can offer useful insight into department and program effectiveness when carefully analyzed and interpreted in the context in which it was collected – for overall program improvement. Data are misleading, and even threatening, when they are used for purposes other than originally intended and agreed upon. For example, data from assessment of student performance in a capstone course should be used to identify areas of strengths and weaknesses in student learning across the students’ entire experience in the major. In this way, these data guide curricular modifications and departmental pedagogical strategies. These data should not be used to evaluate the performance of the capstone course instructor.
What if the capstone (or culminating experience) won’t work for your program’s assessment? Accordion Closed
If you find that:
- the capstone is not a good place to conduct the assessment, or
- that not all learning outcomes are included in the capstone, or
- you are unable to assess all learning outcomes in the capstone,
Then you will want to take two steps. The first step is to consider revising the capstone, because if it’s not providing a culminating experience for students wherein they demonstrate their learning from throughout the program, then it’s likely not achieving its goals.
The second step is to use the curriculum map to identify the course or experience within which you will conduct the (or an additional) assessment(s). The curriculum map makes it possible to identify where, within the current curriculum, your program’s learning outcomes are addressed. If you need to use the map to identify a course or experience within which you will conduct the assessment, simply:
- identify the learning outcomes that you need to assess,
- find the course that is addressing that outcome and doing so as close to the end of the students’ experience in the program, and
- pull the syllabus for that course.
- Review the syllabus to determine what, if any, assignments would address the learning outcome. Since the syllabus (based on the syllabus policy) must address the learning outcomes identified in the curriculum map, and be consistent across class sections, the syllabus should provide you with the information you need to determine whether the course’s current design will suffice for incorporating assessment practices.
You may need to work with the faculty members teaching that course to re-design different aspects of it, or re-think the assignments to ensure they are truly representative of the learning outcomes.
Enlist the assistance of assessment and testing specialists when you plan to create, adapt, or revise assessment instruments. Staff in the Office of Curriculum, Learning Design & Academic Assessment (LINK) are happy to assist you in finding the appropriate resources and helping you to design the assessment. Areas in which you might want to seek assistance include:
- ensuring validity and reliability of test instruments;
- ensuring validity and reliability of qualitative methods;
- identifying appropriate assessment measurements for specific goals and tasks; and
- analyzing and interpreting quantitative and qualitative data collected as part of your assessment plan.