How to select or develop direct measures of student learning

How do I select a direct measure of student learning in the curriculum?

Adapted from the University of Massachusetts, Amherst “Program-Based Review & Assessment Tools & Techniques for Program Improvement,” (April 2017).

The most effective assessment approach is one that is closely linked to the curriculum and uses available information and resources to the greatest degree possible. Before designing additional assessment components it’s important to document how the current curriculum matches the program’s learning outcomes and to inventory what assessment-related information/processes are already in place that you can draw upon.

At NAU, we strive to engage academic unit faculty in developing course-embedded assessment in their capstone courses (and comprehensive exams or theses/dissertations in graduate programs), as it has the greatest potential to provide an efficient, effective approach to collecting assessment data. Over the past decade at NAU, faculty and staff have noticed that the more complex an assessment plan is (the more courses and assignments that are used to collect data), the more likely that data will not be collected, interpreted, and used. By identifying one or two places to conduct assessment, it makes the process much more manageable.

Outcome measures should meet three criterion

While learning outcomes describe the knowledge, skills and abilities that students should possess after instruction (or completion of the program), outcome measures are the specific tools and measures that generate data and information about students’ performance relative to learning outcomes. Regardless of the type of measure used, strong measures share three basic qualities:

  1. Provide sufficient data and information to measure the learning outcome;
  2. Are not overly burdensome on faculty or units to collect; and
  3. If possible, have established performance standards and expected results to help guide analyses.

There are many issues to consider when selecting direct measures of learning. Programs should be creative in determining the most useful way to measure student performance. Yet, in this creativity, it is essential that the assessment approaches selected allow faculty to derive meaning from the qualitative or quantitative data generated. Faculty must be able to link the results to the learning outcomes, and interpret the results so that they can use the results to inform assessment decisions, curriculum decisions, and/or the dissemination of learning strengths. Choose assessment methods that allow you to assess the strengths and weaknesses of student learning. Effective methods of assessment provide both positive and negative feedback. Finding out what is working well is part of the assessment goal; the other part of the assessment goal is to find out what isn’t working well.

What are some guidelines for selecting outcome measures?

Adapted from Volkwein, J., Program evaluation and assessment: What’s the question (1996).

Faculty teaching within the academic program will select or develop outcome measures that will provide the most useful and relevant information to determine the achievement of student learning. Not all methods work for all fields of study or are appropriate to all reasons for assessing. Most importantly:

The evidence you collect depends on the learning outcomes you want to examine. In thinking about program learning outcomes assessment, three questions come to mind:

  • Does the program do a good job at achieving the learning outcomes it sets out to achieve?
  • Does the program meet or exceed certain learning standards for their outcomes?
  • How can students’ learning experiences be improved?