Evaluating Your OER Program
Evaluating your program using local data, evidence and case studies that demonstrate your program's successes will strengthen your advocacy efforts. This is a good time to look back at your advocacy action plan goals for a point of comparison and ask yourself:
- What was the original purpose of your program? Did you program meet this purpose, and how can you evidence this?
For example, when the program purpose is to increase the usage of open textbooks at your institution, your report can include the:
- number of open textbook workshops conducted
- attendance (consider demarcating numbers based on discipline or faculty)
- correlation between attendance and open textbook adoptions (are there instances of staff adopting an open text without attending a workshop?)
- open textbook adoptions across the university (consider reporting based on discipline or year of study – were adoptions more prevalent in first-year courses?)
- number of students engaging with open textbooks.
- What expectations did you set for what success looks like?
Using the previous example, you may have set the following as markers of success:
- all first-year lecturers from three specific disciplines will attend an open textbook workshop
- thirty per cent of those attending will set an open textbook in their course
- a learning and teaching event will be held after semester featuring some of these lecturers to help disseminate this practice.
Below are some tools and resources you can use to evaluate your OER program against its intended goals:
- assessment questionnaire [Word 42KB] developed by CUNY OER Research Program for individual teaching staff to evaluate the impact of OER use on pedagogy
- Gallant, J. (2022). Calculating and reporting student savings. In A. K. Elder, S. Buck, J. Gallant, M. Seiferle-Valencia & A. Ashok (Eds.), The OER starter kit for program managers. Rebus Community. https://press.rebus.community/oerstarterkitpm/chapter/chapter-22-calculating-and-reporting-student-savings-2/
- Gallant, J. (2022). Data collection and strategies for OER programs. In A. K. Elder, S. Buck, J. Gallant, M. Seiferle-Valencia & A. Ashok (Eds.), The OER starter kit for program managers. Rebus Community. https://press.rebus.community/oerstarterkitpm/chapter/chapter-21-data-collection-and-strategies-for-oer-programs/ – includes strategies for collecting both quantitative and qualitative
- Nyamweya, M. (2018). A new method for estimating OER savings. SPARC. https://sparcopen.org/news/2018/estimating-oer-student-savings/.
Keep in mind that results and achievements can't always be measured in numbers. Case studies are important for collecting and describing qualitative data like barriers, solutions and complex stakeholder exchanges.