Designing Assessments

Next module

Designing Online Learning from the Ground Up

Module 7

Designing Online Learning from the Ground Up

Glossary

Assessment as defined by George Brown College's Assessment of Student Learning Policy is "Any form of student activity in a course for which a grade is granted by the faculty member." (A006; 2019-09-01)

As stated in Giving Students Feedback "the presence — or absence — of a mark or grade is what clearly differentiates the concepts of evaluation and feedback." The focus of this booklet is for assessment - evaluation where a grade is assigned for a body of work.

Summative Assessment describes how we examine student learning, usually within the context of, or weighed against stated outcomes. Thus, the assessment OF learning is critical here. This description of assessment is the one used most interchangeably with the term Evaluation, and is where evaluation and assessment overlap. It is used to gather information on how well the student has achieved the outcomes set out in the course and details achievement of the outcomes.

Evaluation describes a process whereby performance is judged against a standard. In our context, the performance of students on a variety of measures are rated against the outcomes laid out by individual courses.

Outcomes Based Assessments allow students to demonstrate their achievement of learning outcomes in ways that are relevant, evidence based and accommodate the increasing diversity of learners. Providing a variety of assessment methods within a course increases the likelihood a larger proportion of students will have opportunities to effectively demonstrate their learning.

Driscoll & Wood (2007). Developing Outcomes-based Assessment for Learner Centered Education. Sterling, V.A: Sylus

Feedback provides students with information about their progress in learning – it does not need to be tied to a mark or a grade. Formal feedback provides information to students specifically related to the course learning outcomes. It can be verbal or written. Informal feedback is less directly attached to a specific learning outcome. This kind of feedback is most often verbal and usually occurs in the classroom. It can be as simple as the teacher recognizing an intriguing question.

Self-Assessment is the process whereby students consider their own work within the context of a given task.

Rubrics/Marking Criteria are tools that can be used to explicitly outline what is expected of a given task. They can be used in evaluation with scores assigned to each category of expectation and/or level of expectation. The following link provides more information on different types of rubrics.

Authentic Assessment describes a particular model of evaluation that requires students to perform real-world tasks to demonstrate meaningful application of the expected knowledge and skills. The guru of authentic assessment, Jon Mueller described "a process in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills"

Universal Design for Learning (UDL) is based on the work of architect Ron Mace who pioneered the term Universal Design, to describe access to buildings for people with varying abilities. The UDL term is used within the education field to describe ways in which we can ensure there are:

Multiple means of representing information
Multiple means of expressing knowledge
Multiple means of engagement in learning for our students

Guiding Principles for Individual Assessments

You can find more detailed explanations on each of these 5 points in the Grading Students Work resource:

Be clear about your expectations
Match your expectations to the learning outcomes
Make authentic assessment tools count
Involve students
Reflect and revise

In addition.....

Consider the assessments you currently use in your courses and the purposes they achieve. (e.g.: Why do you assign a term paper? What are the learning outcomes of that term paper? Is it time well spent (too much or too little according to the importance of its learning outcome?)
Are you conscious of workload for both your students AND yourself? What is the turn around for the chosen assessment and when do students need to get the results for their own assessment /reflection and learning before moving on to the next assessment?
Does practice and feedback occur prior to assessment and grading?

Guiding Principles for Designing a Course Assessment System

You can find more detailed explanations on each of these 5 points in the Grading Students Work resource:

Include assessment tools for all the course learning outcomes and relevant Essential Employability Skills (EES)
Take a Universal Design for Learning Approach: use multiple means of action and expression
Ensure consistent evaluation across sections of a course
Follow College policies and guidelines
Reflect and revise

In addition.....

What elements make up your overall assessment philosophy? (do students get to try an assignment or test a second time?)
Is every outcome assessed at least once, preferably more than once, in a variety of ways?
If an outcome is at an “application” or “analysis” level, does the assessment require the learner to “apply” or “analyze”?
How do you ensure there are no "irrelevant constructs" in your assessments - are you only assessing the stated outcome rather than an underlying requirement for successful completion of the task? *Based on information found on https://ctl.ok.ubc.ca/teaching-development/classroom-practices/assessment/
Design the course around assessments and not around content. The diagram below helps to demonstrate the cycle of creating assignments that are learning outcome driven. Once the learning outcomes are determined, the assessment flows from those outcomes. The content or learning experiences should always flow from the proposed assessments rather than as stand alone "experiences". Thus, you plan the learning activities, teaching methods and resources required to achieve those outcomes and assessments – in turn, each of these elements supports the intended learning outcome(s).

Three arrows in a circle pointing clockwise, with the following text between each arrow: 1. Course Outlines. 2. Assessments. 3. Learning Experiences.

Credit: Learning Outcomes Assessment: A Practitioners Guide (funded by Higher Education Quality Council of Ontario)

Assessment Challenges for Students & Faculty

The following are a collection of ideas/issues gathered from STUDENTS AND FACULTY about what they consider the challenges of assessment. This resource attempts to give some ideas for reducing these difficulties- for both students and faculty.

We encourage you to look at these lists and discuss possible solutions with your colleagues.

Students say:	Faculty say:
The emotional impact of being "judged or criticized", or getting a failing assessment is really hard.	Marking is hard – there is often so much of it, and it's hard to know how to do it well without hurting students. The emotional load/labour of evaluating work can be very difficult.
The timing of evaluation and subsequent feedback doesn't allow me to "correct" mistakes I've made if the feedback comes AFTER the next assessment is due.	How do I ensure the assessment actually measures the attainment of an outcome?
Unclear expectations on what is needed in the assessment (rubrics or clear outlines are not given to me along with the assessment)	There never seems to be enough time for assessment (classes size increasing) and the different abilities and needs of students seems to be changing.
What is being assessed? I'm being asked to present knowledge but the outcome specifies that it has to be done in writing? Why am I being assessed on writing - it's not an outcome of the course?	Academic Integrity – plagiarism is a real issue. I spend a lot of my time helping students understand what constitutes cheating. I then spend a lot of time creating assessments that I hope are "cheat proof".
When different faculty teach the same course, often really different marks are assigned. I would like to see consistency of assessment.	I struggle with how to make sure group/team work is assessed so some students don't do most of the work while others "ride"; yet I assign the same grade.

Assessment Success Strategies

Here are some ideas of what other faculty have designed to help their students demonstrate the achievement of Learning Outcomes:

Posting Exemplars

The faculty member in this course scans copies of some of the best papers from the previous semesters' students. They use these papers in class to demonstrate how they have met or exceeded the assessment expectation of the rubric that was created. By showing the current students how the paper meets each of the elements of the rubric, they reinforce the expectation of the assessment and gives students the opportunity to see, ask questions and discuss an exemplar of the assessment.

Authentic Assessments

Assessments will vary across the differing disciplines; here are a few to get you started thinking about incorporating authentic assessments into your course:

In the Personal Support Worker program, the students create a budget for meals for a week using 10 discount coupons from newspapers.
The Horticultural Technician program has students produce a plan for a small city garden using native flowers.
In the Community Worker program students are expected to run a meeting according to Robert's Rules.
The Registered Practical Nurse program encourages students to develop a blog of their weekly progress in a clinical nursing setting.
The Activation Coordinator/Gerontology programs asks students to create a slide show on the dietary needs of the elderly for a community agency.
An Introductory Sociology course allows students to create a story board of their learning in the course and highlight five key concepts.

Student Workload Map

The faculty team in Program XXX had been receiving feedback from their students about heavy workload for a number of years. They asked for some curriculum help and worked with a specialist from outside their program on an "assessment map. It was an eye opening experience to say the least. They could see weeks with no assessment in any course were countered by weeks where students had up to 4 assessments due on any one day. This exercise helped them to discuss and work together to balance the need for assessment in their individual courses with the needs and stressors placed on students.

Marking in Blackboard

The different assessment options run from standard tests, assignments, self and peer assessments; to journals, wikis, blogs, and portfolios. In addition, a great way to foster connection and further deeper exploration of an issue is through graded discussions. Explore what’s available in the Assessment and Tools menus on the Blackboard interface. You can also contact the Teaching and Learning Exchange (TLX) for hands-on help. The TLX also offers newsletters with tip and ideas to help you.

Blackboard has lots to offer all faculty members who are looking to add to their teaching toolkits and not just those who teach online courses. Blackboard can provide a medium through which teachers can develop (and evaluate) different assessments to meet the needs of our diverse learners.

Assessment Across Courses

Two faculty who teach in the same program came up with the idea for a "joint" assessment. One faculty taught the Introductory Sociology course while the other faculty taught the COMM1007 course. They designed a research paper that was grounded in the outcomes for both courses. They created marking criteria that outlined their clear expectations for each course. Students submitted the same paper to both faculty and received an assessment (grade) based on the criteria. Students appreciated this collegial approach to the research paper - two assessments with one paper.

Assessment using UDL

The faculty in this course creates assessments that are varied and allow students the opportunity to "choose" how they will demonstrate their achievement of a specific outcome. The assessment rubric of the outcome(s) remains the same, however students can opt to write a traditional essay on a given topic or opt to do a podcast or video demonstrating their understanding of the material. Some students do a weekly journal (either written or videoed) on topics discussed in class. This allows students the opportunity to demonstrate their knowledge of the subject area and their achievement of the outcome(s) in ways that are meaningful and achievable for them.

Tips for Assessment

First, it is important to acknowledge the pain of judgment for faculty when assessing students. For some of us, procrastination is an emotional regulation issue - looking at that stack of papers can be daunting. Trying strategies where you create stacks of 5 papers and once completed, make yourself a cup of tea... or have an indulgent snack. This marking reward can help break up the task. In addition, giving yourself time to stand up, stretch, take a walk around your space or do some stretches all help to break up the marking regime and makes you "fresh" for the next round of assessments.
Creating assessments that are "cheat proof" can be a challenge. Some faculty create assessments that ask students to draw a direct connection between discussions in class along with new material the student has sourced outside of class. This means general papers or articles that are found on the Internet cannot be cut and pasted into an essay. The student must demonstrate an analysis of the course material (an outcome of the course) while still having some creativity in linking that work to an article they have found elsewhere (this also credits an EES that is evaluated in that same course)
Involve your students in creating and answering problem based assessments. They come to class with a scenario that also includes a potential answer to the problem. You collect their submissions and part of the assessment is their answer for their own scenario. The following week, you re-distribute the problems and students have the opportunity to answer a peer's scenario. The second part of the grade is given for their solving of the peer's scenario.
By using the principles of Universal Design for Learning (UDL) you can create assessments that are both accessible for students and fun to mark. The agony of sitting in front of 65 identical assignments can be mind numbing. Be creative and design assessments that allow choice for students. Yes, the outcome(s) still needs to be met, but varying the vehicle that students demonstrate their attainment of the outcome can have great variety. Include skills demonstrations, written assignments, oral presentations, group projects, quizzes, videos, posters, student choice, etc.
Multiple Choice Quizzes can be an effective way of students demonstrating problem solving, analysis and critical thinking (not just rote learning). The important thing is to structure the questions so students have to use these higher level functions to answer the questions. David DiBattista of Brock University is a good resource and starting point for exploring how you can make challenging and effective multiple choice questions.
Speed Dating is an interesting and "easy to mark" idea that uses class time. Students present short summaries or explanations while you and the class rotate through and "mark on the spot". As with any "easy fix", there has to be time spent preparing students on giving feedback. Ron Berger's mantra is a great place to start...be kind, be specific, be helpful.
In practical courses, students can videotape demonstrations and send them to the faculty (there are technologies for iPads that allow the faculty to mark within a rubric while viewing the video).
As stated in the Glossary section, rubrics/marking criteria can be used in assessment with scores assigned to each level of expectation. Here are two on-line resources to help you create rubrics: Rubistar and IRubric.
TLX offers support on Team/Group Work. You'll get lots of great resources and ideas there. Some ideas could include:
- Creating group contracts,
- Breaking the project into steps and checking in with groups at each of the steps,
- Allocating the grade to each group member based on peer and self assessments,
- Ensuring the work should be done in a group…thus, part of the assessment is how well they worked as a group (process and product),
- Giving time in class for group work so students don’t have to coordinate outside class,
- Encouraging the use of technology for group members to meet virtually using software like Teams. Faculty can also "drop in" for a Teams meeting to monitor progress,
- Keeping groups small (no more than 4-6),
- Making sure students have learned effective strategies for working in groups before the skills are expected to be demonstrated.
Here's a checklist for some assessment tips using Blackboard. Many of these tips can be useful in face-to-face or blended courses as well:
- Have a practice version of an assignment students can complete, so they can become familiar with the technology. Don’t assume students will know how to complete or submit an assignment using Blackboard, so give them something to practice.
- Try to avoid online “objective” assessments (usually tests where there’s only one right answer) that are high value. Not only does this try to safeguard against cheating, it’s also a best practice of assessment in general. This all means that online tests are “open book.”
- Use a variety of assessment methods. Don’t rely too heavily on just graded discussions or on tests.
- When using the Blackboard assignment tool, provide written feedback electronically so students can see where they did well and where they need to make improvements.
- It takes time to set up, but using the rubric function on Blackboard can save you time in the end. You can find lots of Blackboard rubric templates by doing a simple Internet search, so you don’t have to invent rubrics entirely from scratch. One point rubrics are also emerging as a good alternative and take less time to set up.
- Feedback you give students should be precise and timely. If you’re teaching fully online, feedback should be more frequent than face-to-face teaching. It’s also a good idea to allow students a chance to provide feedback to each other that is constructive and developmental.
- Online discussions are a great way to get students to interact with each other who might not otherwise connect in class. Graded discussions give students an incentive to participate, and they can learn and get feedback on a different set of skills. Don’t forget to provide clear expectations for what online discussions entail.
- Finally, always allow opportunities for students to revise their work.
- Here's a link to Blackboard training modules that will help you navigate, plan and create assessments.