This is a short version of the more detailed post here.
Your course’s learning outcomes are likely tied to a “graduate attribute indicator” GAI. The learning outcome is specific to your course, but the GAI is applicable across both the program and the entire Lassonde school. When looking to assess your learning outcome, examine the GAI for context and try to frame your learning outcome in terms of the associated GAI.
The following illustrates three examples of assessments that took both the GAI and the learning level of the students. In 1st year, the students are typically taking courses that are “I” (Introductory) in nature. In second and third years, the courses are typically “D” (Developed), while in 4th year (and sometimes 3rd), the courses are typically “A” (Advanced or Applied).
Here I try to show what these types of questions could look like at the three levels.
The graduate attribute indicator examined here is “5a: Select and adapt appropriate equipment and tools to perform tests or measurements”. It’s part of CEAB’s 5th graduate attribute category on “Using Engineering Tools”.
“Introductory” level version (1st Year):
Question: In Lab F, you used a device to perform a measurement. Keep each answer to less than ten words. One point per question. No part marks. (a) What was it that your were measuring in Lab F? (b) What kind of device did you use to undertake the measurement? (c) What were the units of this measurement? (d) What range of values were you expecting to observe while performing your measurement?
Answer: (a) Voltage across a battery. (b) a Fluke multimeter (c) Volts (d) 0 to 20 Volts.
Grade: three out of three
Notes: The question was broken up into individual parts that allow for distinct but related pieces of knowledge to be expressed and assessed. This helps both you and the student during the assessment.
“Developed” level version (2nd Year):
Question: In Lab B, you were asked to measure voltage in a mixed signal circuit but were not told which tool to use. What tool(s) did you use to measure the voltage and why? Were there any significant adjustments needed on any of your tools? You will be evaluated using the Rubric for GAI 5a, as shown below. Please note that in such a rubric-style of assessment, a score of 3/4 (“meeting expectations”) is typical.
Answer: I used a standard analogue oscilloscope to measure the analogue signal voltage, both immediately after the DAC and after the amplification stage. However, on the digital input of the DAC I used a Logic Analyzer with I2C decoding capability.
Grade: 3 out of 4: Meeting expectations, as per the rubric below. The student used the tools as expected.
“Advanced” level version (4th Year)
Question: In your final project, (a) describe the equipment that you used to measure the performance of your system. (b) What were the criteria that led you to choose that equipment? and (c) if you had to adjust or adapt the measurement equipment to improve the measurements, why did you do so and why? If you did not, simply state that you did not. Note that you will be graded using the following rubric. Most students are expected to get “Meeting expectations” (numeric grade: 3 out of 4)
Answer: We used a Model XYZ logic analyzer to examine the serial data stream from our embedded microcontroller as it communicated with a sensors on over RS232. We chose the Model XYZ logic analyzer because it could capture low current digital signals with voltages from 0-3.3v at data rates of up to 20 MB/sec. Using the logic analyzer, we determined that the actual baud rate was within 2% of the desired value but we had to write a decoder script for the logic analyzer to automatically decode the packets as the included decoders were insufficient.
Grade: Student score: 4/4 (“exceed expectations”) because the student met all the typical expectations and also wrote the optional custom decoder to aid in their analysis.
Notes: Use the rubrics to qualitatively evaluate the student answer. Give 0 for no answer, 1 for “Below expectations”, 2 for “marginally meeting expectations”, 3 for “meeting expectations” and 4 for “exceeding expectations”. Don’t give part marks. Ideally, most students will get 2 or 3. Few should get 4, otherwise your expectations should be recalibrated. Likewise if most students are getting 0 or 1, then a recalibration of the assessment or the formative material should be considered.
As instructors we design assessments. Above, we looked at how characteristics like the GAIs and the IDA levels can be incorporated explicitly into your assessment design process. Once the semester is done, have a look at how the students performed on the assessment. If you used a rubric, look at the result: did most students exceed the expectations? If so, does this indicate that a recalibration of the assessment (or of the lessons that led to it) is in order?
James Andrew Smith is a Professional Engineer and Associate Professor in the Electrical Engineering and Computer Science Department of York University’s Lassonde School, with degrees in Electrical and Mechanical Engineering from the University of Alberta and McGill University. Previously a program director in biomedical engineering, his research background spans robotics, locomotion, human birth and engineering education. While on sabbatical in 2018-19 with his wife and kids he lived in Strasbourg, France and he taught at the INSA Strasbourg and Hochschule Karlsruhe and wrote about his personal and professional perspectives. James is a proponent of using social media to advocate for justice, equity, diversity and inclusion as well as evidence-based applications of research in the public sphere. You can find him on Twitter. Originally from Québec City, he now lives in Toronto, Canada.