![\"A](\"https:\/\/in.nau.edu\/professional-education-programs\/wp-content\/uploads\/sites\/367\/2018\/07\/QAS-infographic-Color-600x600.jpg\")
<\/p>\n<\/p>\n
A Quality Assurance System is composed of multiple elements such as Analysis & Assessment Planning, Assessment Policies & Procedures, and Data Tools & Human Resources.\u00a0 The purpose of a quality assurance system is to demonstrate alignment to overarching standards or learning outcomes and develop an efficient, sustainable process for systematic review of assessment instruments and data for continuous improvement of program and\/or the unit.<\/p>\n
The following PowerPoint presentation was developed for a workshop at the 2017 American Association of Colleges of Teacher Education (AACTE) Quality Support Workshop \u2013 Midwest.<\/p>\n
Conn, C., Bohan, K., & Pieper, S. (2017, \u00a0August).\u00a0Developing a quality assurance system and examining the validity and reliability of performance assessments<\/a>. Presentation at the annual conference of the American Association of Colleges of Teacher Education (AACTE) Quality Support Workshop \u2013 Midwest, Minneapolis, MN.<\/p>\n \n\n \n Several strategies can be used to assist with designing a comprehensive Quality Assurance System. These include:<\/p>\n Following are resources and templates related to the strategies.<\/p>\n \n\n \n The following PowerPoint presentation provides an overview of the Validity Inquiry Process (VIP):<\/strong><\/p>\n Conn, C., Bohan, K., & Pieper, S. (2018, April). Ensuring meaningful performance assessment results: A reflective practice model for examining validity and reliability.<\/a> Presentation at the annual conference of the American Educational Research Association, New York City, New York.<\/p>\n Bohan, K., Conn, C. A., & Pieper, S. L. (2019). Engaging faculty in examining the validity of locally developed performance-based assessments. In K. K. Winter, H. H. Pinter, & M. K. Watson (Eds.), Performance-based assessment in 21st century teacher education<\/em> (pp. 81-119). doi: 10.4018\/978-1-5225-8353-0.ch004<\/p>\n The Validity Inquiry Process (VIP) is intended to assist in gathering evidence to build a validity argument regarding performance assessment(s). The process is aligned to eight validity criteria outlined in the literature (Linn, Baker, & Dunbar, 1991; Messick, 1994): 1) Domain Coverage, 2) Content Quality, 3) Cognitive Complexity, 4) Meaningfulness, 5) Generalizability, 6) Consequences, 7) Fairness, and 8) Cost and Efficiency.<\/p>\n The VIP includes practical resources, including strategies and instruments, for examining locally developed performance assessments. The purpose of these strategies and instruments is to gather evidence for building a validity argument for the interpretation and use of the performance assessment results. The instruments developed to guide the validity inquiry process include:<\/p>\n Kane, M. (2013). The argument-based approach to validation. School Psychology Review, 42(4), 448-457.<\/p>\n Linn, R. L., Baker, E. L., & Dunbar, S. B. (1991). Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20(8), 15-21.<\/p>\n Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23.<\/p>\n Pieper, S. L. (2012, May 21). Evaluating descriptive rubrics checklist (pdf<\/a>)<\/p>\n Do not reprint information contained or linked from this website without written permission of the authors. The inter-rater agreement and calibration strategies described on this page are a component of the Quality Assurance System (QAS) implemented by NAU Professional Education Programs (PEP). NAU PEP will continue to put in place these strategies and may make changes to the process at any time.<\/p>\n Important note about these files: The Excel spreadsheets contain password-protected cells. If you’d like to start from scratch and modify the files, please create a copy of the file on your computer. The Final Report PDF is a secured document to prevent editing.<\/p>\n Fall 2016 CAEPCon CWS Evaluator Training Inter-rater Agreement Template (Excel<\/a>) Graham, M., Milanowski, A., & Miller, J. (2012). Measuring and promoting inter-rater agreement of teacher and principal performance ratings<\/em>. Washington, DC: Center for Educator Compensation Reform. Retrieved from https:\/\/files.eric.ed.gov\/fulltext\/ED532068.pdf<\/a><\/p>\nWithin the tabs below are a variety of resources and templates.<\/h2>\n\n\n\n
\n \n
\n \n Quality Assurance System Tab Open<\/span>\n <\/h4>\n <\/div>\n <\/a>\n <\/li>\n
\n \n Validity Inquiry Process Tab Closed<\/span>\n <\/h4>\n <\/div>\n <\/a>\n <\/li>\n
\n \n Inter-rater Agreement and Calibration Strategies Tab Closed<\/span>\n <\/h4>\n <\/div>\n <\/a>\n <\/li>\n <\/ul>\n \n
\n \n \n Quality Assurance System Accordion Open<\/span><\/h4>\n <\/span>\n <\/div>\n <\/a>\n
\n \n\n\n![\"Quality](\"https:\/\/in.nau.edu\/professional-education-programs\/wp-content\/uploads\/sites\/367\/2018\/07\/QAS-Strategies-infographic-Color-600x600.jpg\")
\n <\/span>\n <\/span>\n<\/p>\n\n
\n
\n \n \n Validity Inquiry Process Accordion Closed<\/span><\/h4>\n <\/span>\n <\/div>\n <\/a>\n
\n \n\n\n![\"Validity](\"https:\/\/in.nau.edu\/professional-education-programs\/wp-content\/uploads\/sites\/367\/VIP_Diagram_Color_2500x-600x462.png\")
\n <\/span>\n <\/a><\/span>\n<\/p>\nOverview<\/h3>\n
Resources<\/h3>\n
\n
\n
\n
\n
Video Examples of the VIP Facilitated Meetings<\/h3>\n
\n
References<\/h4>\n
\n<\/p>\n\n <\/body><\/html>\n\n <\/div>\n<\/div>\n\n \n\n \n \n Inter-rater Agreement and Calibration Strategies Accordion Closed<\/span><\/h4>\n <\/span>\n <\/div>\n <\/a>\n
\n \n\n\nInter-rater agreement training:<\/h3>\n
\nFall 2016 CAEPCon Expert Panel Scoring Sheet for Calibration Session Template (Excel<\/a>)<\/p>\n