OSEP Hosts National Summit and Launches New Attract, Prepare, Retain Resource Page
The 2020 OSEP Summit provided an opportunity by bringing together various stakeholders to explore potential strategies and innovative approaches to address this critical need. The virtual event was held over three days and participants had the opportunity to register for each event individually or attend all three. An archive of the event can be found on the 2020 OSEP National Summit webpage. OSEP also launched a new Attract, Prepare, Retain Resources Page that provides research and resources for stakeholders to explore potential strategies, innovative approaches, and additional resources regarding this topic!
This page will be updated with additional resources on planning and conducting evaluation activities. You can also click "Logic Models and Performance Measures" to view information from OSEP on laying the groundwork for a comprehensive evaluation, and high quality objectives and performance measurement.
Compared to What? Identifying Good Comparison Data to Assess Project Results: Measuring outcomes for the population affected by your project is important, but measuring outcomes in isolation does not tell the full story. You need to consider how the outcomes may have been different if your project had not been implemented. That is, you need to know, “compared to what?” Below are each of the briefs that make up the 5-part series to help grantees identify good comparison data to assess project results.
- Compared to What? Identifying Good Comparison Data to Assess Project Results
- Compared to What? Using Extant Data
- Compared to What? Using Nonequivalent Pre-Post Control-Group Designs
- Compared to What? Using One-Group Pre-Post Designs
- Compared to What? Using Single Case Designs
Linking Expectations to Evaluations: Using Your Logic Model to Create Your Evaluation Plan: This brief provides guidance and examples for creating strong alignment between project evaluation plans and logic models. This will help you to target evaluation resources and focus your energy. This document can be used in conjunction with the other resources listed on this page to ensure your logic model and evaluation plan are aligned, accurately reflect your project, and are of high quality.
Making Project Measures Meaningful: Quality, Relevance, Usefulness, and Beyond: OSEP grantees often focus their evaluations and project performance measures on the outcomes of quality, relevance, and usefulness (QRU), consistent with the Government Performance and Results Act (GPRA). In this brief, we focus on developing project performance measures that move beyond assessments of QRU in two ways, by: (1) Limiting the number of project performance measures focused on QRU and (2) Striving for a greater focus on other types of outcomes in your remaining measures. These shifts can open the door to incisive measures that better capture your important project work and the outcomes associated with it.
Hiring a Third-Party Evaluator: This brief gives a quick overview of important factors to consider when hiring a third-party evaluator to support your project.
Managing Evaluation Data: This brief outlines the importance of managing your evaluation data and provides some tips for successful data management.
Grantee Guide to Project Performance Measurement: This CIPP TA product is designed to help projects funded under OSEP’s discretionary grant programs develop high-quality project performance measures. These project-specific measures supplement the required Government Performance and Results Act (GPRA) measures grantees are required to report to OSEP each year. This guide presents an approach for developing high-quality project performance measures and explains how the GPRA and project measures work together to allow for meaningful tracking of program and project performance. The guide also outlines a simple, but thorough, five-step process for developing performance measures that:
supports reviews of documentation relevant to project work;
helps identify critical activities, outputs, and outcomes associated with the project; and
guides selection of the best measures of project performance.
Templates for each step are provided below.
Grantee Guide to Project Performance Measurement Webinar: In this webinar staff from the Center to Improve Program and Project Performance (CIPP) talk about the CIPP Grantee Guide to Project Performance Measurement, which presents a thorough, yet simple, approach to identifying and creating high-quality project performance measures. The tool guides users through the process with the goal of helping them demonstrate credible evidence of project progress and results. The tool includes a number of templates, including Excel worksheets, that grantees can use to develop their own project performance measures.
Effectively Communicating Evaluation Findings: This CIPP TA product is designed to help grantees effectively communicate information about project progress and results to a variety of audiences. The product includes a discussion of why grantees might want to communicate with a specific audience, and outlines key questions to ask when designing a communications plan for different audiences. It highlights some contextual factors that might influence communications with specific audiences, and presents a brief discussion of challenges and ethical considerations associated with communicating evaluation findings. It also provides an overview of common communication tools and products that have been found useful with specific audiences. Throughout, checklists and worksheets tailored to specific audiences are included to help staff think through how to create a plan to link evaluation data and communication strategies.
Evaluating Special Education Programs: Resource Toolkit: This comprehensive Toolkit provides an array of evaluation resources for OSEP grantees. Starting with a discussion of the basics of evaluation, including working with third-party evaluators and developing an evaluation budget, it walks the reader through the process of planning and conducting an evaluation. It also offers some key methodological considerations, such as how to conduct high-quality data analysis and how to develop a system to measure fidelity of a project. The Toolkit is designed to highlight important issues to consider and to offer a variety of resources, including sample templates and forms, links to other evaluation-related products and resources, and recommended readings on research and evaluation.
Demonstrating Evidence across the Project Cycle: This interactive tool discusses the different types of evidence that can be used throughout the project cycle to, for example, support theories of action, choice of interventions, and provide formative feedback on how project activities are progressing. It also discusses how to use the evidence collected to monitor progress and fidelity in implementation with the goal of demonstrating ultimate project results, or long-term outcomes. For each phase, the tool identifies what types of evidence might be collected, where they might be found, or how they might be produced and used. The tool includes links to resources that grantees and evaluators might use when evaluating their project.
CIPP Logic Model Outline: The CIPP Logic Model Outline provides information to help project staff and evaluators think through how they will implement and evaluate their project. It provides guidance on typical types of inputs, activities, outputs and expected short-term, medium-term, and long-term outcomes that should be considered when planning, implementing, and evaluating a project. It also illustrates the roles of external factors and evaluation throughout the project cycle.
Budgeting for Evaluation Brief: Knowing how much to budget for an evaluation requires an understanding of the evaluation process and of the various factors that might influence costs. This CIPP brief discusses these factors and the included Evaluation Cost Considerations Worksheet can help you think more deliberately about the different factors affecting costs in an evaluation.
Conceptualizing Capacity Building: Capacity building is a term that is frequently used, yet hard to define and measure. The CIPP brief “Conceptualizing Capacity Building” presents the different dimensions of capacity building described in the literature and outlines strategies to evaluate capacity building efforts. It also includes an annotated bibliography with links to useful tools such as state and district capacity assessments.
Why Evaluate?: This CIPP infographic discusses some of the reasons to conduct an evaluation and outlines concerns that are commonly associated with evaluations.
Grantee Guide to Project Performance Measurement
In this webinar staff from the Center to Improve Program and Project Performance (CIPP) talk about the CIPP Grantee Guide to Project Performance Measurement, which presents a thorough, yet simple, approach to identifying and creating high-quality project performance measures. The tool... Read More
Demonstrating Evidence Across the Project Cycle: A New Resource for OSEP Staff and Grantees
During the webinar, CIPP staff will present the new tool, Demonstrating Evidence across the Project Cycle, which discusses the different types of evidence that can be used throughout the project cycle to monitor progress and fidelity in implementation and to demonstrate results. Participants... Read More
Two-Part Webinar Series on Planning and Conducting Qualitative Interviews
These webinars discuss the ins and outs of how to plan and conduct high-quality customer interviews as part of a rigorous evaluation. The first webinar discusses the benefits and limitations of using qualitative interviews in an evaluation and walks participants through the “who, what, when,... Read More
Guidelines for Working with Third-Party Evaluators
Are you looking for a third-party project evaluator? Find out what to look for, why you need a third-party evaluator and how to plan for their work with your project.
This Exploration/Application webinar focuses on identifying evaluation needs, finding and hiring, as well as monitoring... Read More
Three-Part Webinar Series on Customer Survey Development
These webinars discuss important aspects related to planning, designing and conducting customer surveys as part of an evaluation. The first webinar outlines the varied purposes of customer surveys, walks participants through the process of planning a high-quality customer survey, identifies some... Read More