Direct methods of assessment

Direct measures of student learning require students to display their achievement of student learning (knowledge and skills) as they respond to the assignment, or “instrument,” itself. Because direct measures capture what students can actually do, they are excellent for measuring levels of achievement of student learning on specific outcomes.

What are “direct” and “indirect” measures of student learning, and why do we focus on direct measures at NAU?

Adapted from Marymount University Assessment Handbook (2015), from the University of Massachusetts, Amherst “Program-Based Review & Assessment Tools & Techniques for Program Improvement,” (April 2017), and from Western Washington University’s Tools & Techniques for Program Improvement: Handbook for Program Review & Assessment of Student Learning (2006), and from Ball State University Assessment Workbook (2012)

Following are the definitions of direct and indirect measures of student learning:

Direct measures “require students to display their knowledge and skills as they respond to the instrument itself. Objective tests, essays, presentations, and classroom assignments all meet this criterion.”

Examples of Direct Measures of Student Learning from Cleveland State University Accordion Closed
  • Scores and pass rates on standardized tests (licensure/certification as well as other published tests determining key student learning outcomes)
  • Writing samples
  • Score gains indicating the “value added” to the students’ learning experiences by comparing entry and exit tests (either published or locally developed) as well as writing samples
  • Locally designed quizzes, tests, and inventories
  • Portfolio artifacts (these artifacts could be designed for introductory, working, or professional portfolios)
  • Capstone projects (these could include research papers, presentations, theses, dissertations, oral defenses, exhibitions, or performances)
  • Case studies
  • Team/group projects and presentations
  • Oral examination
  • Internships, clinical experiences, practica, student teaching, or other professional/content-related experiences engaging students in hands-on experiences in their respective fields of study (accompanied by ratings or evaluation forms from field/clinical supervisors)
  • Service-learning projects or experiences
  • Authentic and performance-based projects or experiences engaging students in opportunities to apply their knowledge to the larger community (accompanied by ratings, scoring rubrics or performance checklists from project/experience coordinator or supervisor)
  • Graduates’ skills in the workplace rated by employers
  • Online course asynchronous discussions analyzed by class instructor

Indirect methods such as surveys and interviews ask students to reflect on their learning rather than to demonstrate it.
From Palomba and Banta, 1999, Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education.

Examples of Indirect Measures of Student Learning from Ball State University: Accordion Closed
  • Course grades and grade distributions
  • Assignment grades, if not accompanied by a rubric or scoring criteria
  • Retention and graduation rates
  • Admission rates into graduate programs and graduation rates from those programs
  • Scores on tests required for graduate level work (such as the GRE) that evaluate skills learned over a lifetime
  • Quality and reputation of graduate programs into which alumni are accepted
  • Placement rates of graduates into appropriate career positions and starting salaries
  • Alumni perceptions of their career responsibilities and satisfaction
  • Student feedback of their knowledge and skills, and reflections on what they have learned over the course of their program
  • Questions on end-of-course student evaluation forms that ask about the course rather than the instructor
  • Student, alumni, and employer satisfaction with learning collected through surveys, exit interviews, or focus groups
  • Student participation rates in faculty research, publications, and conference publications
  • Honors, awards, and scholarships earned by students and alumni

Assessment and Grading: Why are grades indirect measures of student learning? Accordion Closed
Grades (including pass/fail) are global evaluations that represent the overall proficiency of students. Traditional grading which offers one “score” to represent the sum total of a student’s performance across a host of outcomes and assignments within a course does not provide the detailed and specific information necessary for linking student performance to program objectives and, ultimately, to improvement. Because grades don’t tell you about student performance on individual (or specific) learning outcomes, or how students are or are not performing well on that outcome, they provide little information on the overall success of the program in helping students attain specific and distinct learning objectives of interest.

A key part of deciding on what assessment methods to use is to know what learning outcome(s) you want to assess. With NAU’s assessment requirement, it is simple to know what to assess: each of your program’s broad learning outcomes.

Since direct measures “require students to display their knowledge and skills (Palomba and Banta, 1999, pp. 11-12),” it is the most effective way to determine student achievement of degree program learning outcomes. Direct measures ensure faculty can assess student mastery of outcomes, which is the focus for program-level learning outcomes assessment.