Unit testing is really important in general programming, but it's also really useful when creating interactive programming assignments. With Java, I like to use JUnit and with C I like to use the Unity unit tester.
With Verilog it's actually really straight-forward to set up testing frameworks. In the EECS 2021 (Computer Organization) labs written by the fabulous Dr. Hamzeh Roumani, we call them "oracles". Because of how it's easy to separate out the logic of the modules we're creating in Verilog from the testing frameworks, we can set up VPL activities that allow us to test only single modules that the students write, while keeping the testing framework in the background.
When setting up a VPL activity for Verilog in the manner described below, note that it is assumed that you are doing verbatim, repeated testing... nothing randomized. The testbench.v file will be run on the teacher solution, teacherDesign.v, and the StudentTemplate.v file should output exactly the same result when the testbench.v is run on it. Students will not see the testbench.v file.
Steps to set up a VPL activity in Verilog with a hidden testing framework:
- Create a VPL assignment in Moodle / eClass. Set max grade to 1 and set completion tracking to "upon grading"
- Hide it.
- Point the VPL assignment to your local VPL server (https://xxxxx.eecs.yorku.ca) inside of the "Local Execution Servers" category. Make sure to https:// it. End with end_of_jails
- Note: send me or EECS tech an email to know what the actual name of the EECS server is. We don't name it here (on purpose).
- Execution options: leave as default execept:
- Run to Yes
- Evaluate to Yes
- Automatic Grade to Yes.
- Create a "Requested Files" called StudentTemplate.v
- Fill in the "vpl_run.sh" file in the "Execution Files" category.
- Fill in the "vpl_evaluate.sh" file in the "Execution Files" category
- Create the "testbench.v" and save it in the "Execution Files" section.
- Create the "teacherDesign.v" and save it in the "Execution Files" section. This is the proper design and will be compared internally to the student submission.
- Modify "Files to Keep while Running" to include testbench.v and teacherDesign.v.
- Test out the assignment
- Unhide the assignment.
Note: the vpl_run and vpl_evaluate shell scripts are designed to output grades of 0 or 1. I like to keep the weights really low. Also, those two scripts actually create two other scripts that VPL will run... you don't need to know the details.
When you adopt this approach to your own VPL activities in Verilog, you will only need to modify three files:
- testbench.v. This contains the testing framework, including the "expect" variable that contains the expected logic for the student's work.
- teacherDesign.v is the model solution... and the models contained within will be called by testbench.v.
- StudentTemplate.v contains the original student file. You can make it completely blank or provide some hints in it. It will be tested by VPL and its Verilog output will be compared to the Verilog output of testbench.v + teacherDesign.v.
You are encouraged to test out testbench.v + teacherDesign.v and testbench.v + StudentTemplate.v from the command line :
iverilog '-Wall' teacherDesign.v testbench.v && unbuffer vvp a.out
command line iverilog command
or with EDA Playground.
James Andrew Smith is a Professional Engineer and Associate Professor in the Electrical Engineering and Computer Science Department of York University's Lassonde School, with degrees in Electrical and Mechanical Engineering from the University of Alberta and McGill University. Previously a program director in biomedical engineering, his research background spans robotics, locomotion, human birth and engineering education. While on sabbatical in 2018-19 with his wife and kids he lived in Strasbourg, France and he taught at the INSA Strasbourg and Hochschule Karlsruhe and wrote about his personal and professionalperspectives. James is a proponent of using social media to advocate for justice, equity, diversity and inclusion as well as evidence-based applications of research in the public sphere. You can find him on Twitter. Originally from Québec City, he now lives in Toronto, Canada.