Candice Benjes-Small and Eric Ackermann from Radford University spoke about how they redesigned their assessment process for instruction. They had reached a point where merely counting number of sessions was deemed no longer useful in measuring success.
All librarians had been using a standard student evaluation form that had a four point Likert scale and a single comment box at the end. They found the disconnect between the scores and the comments to be problematic and not useful in making changes. It was decided to modify the evaluation form to ask for qualitative feedback for each question.
The modified evaluation form asks the following three questions
1. I learned something useful for this workshop.
- Strongly Agree: Name one thing you learned from this workshop?
- Agree: Name one thing you learned from this workshop?
- Disagree: How can the workshop be improved?
- Strongly Disagree: How can the workshop be improved?
2. I think this librarian was a good teacher.
- Strongly Agree: What did you like about the teaching?
- Agree: What did you like about the teaching?
- Disagree: What did you dislike about the teaching?
- Strongly Disagree: What did you dislike about the teaching?
3. I would recommend this workshop to someone interested in library research.
- Strongly Agree
- Strongly Disagree
They chose a comment based metric methodology for assessment. This is similar to what the University of Virginia Library is doing with their balanced scorecard metrics. "What did you dislike about the teaching?" was chosen as the question to measure. This would allow for the librarian teaching to have something tangible for improving instructional delivery. A target of less than 5% negative comments was set to be the measure for total success. Partial success would be achieved if 5 to 10% of the comments were negative.
- Evidence based
- Allows for goals to be set and measured
- Flexible to measure what you want to know
- Time intensive, especially coding qualitative comments
- Difficult to change evaluation forms if you want to go back and measure another goal
Questions to consider
- What do you want to know?
- How are you going to measure?
- Are you going to focus on evaluation scores (quantitative) or comments (qualitative)?
- What is the target for success?
- Who is going to compile the results?
Their PowerPoint slides are available.