Skip to main content Skip to local navigation
Prof. James Andrew Smith's Blog

Virtual Programming Lab: Basic Examples

Posted on 14 June 2023
A programmer. https://thenounproject.com/search/?q=programmer&i=3308619

Heard about Virtual Programming Lab (VPL) and eClass at York and want to “kick the tires” a little? If you haven’t already, VPL is a plugin for Moodle (eClass at YorkU) that allows instructors to create interactive programming activities for students. Here is some introductory material.

[this may need an update after the update in VPL at York in 2022]

Simple Auto-Grade Examples in Multiple Languages

Simple VPL example in C (2011 standard) using a simple string test to verify correctness.

Let’s do “Hello World” in a few different languages and show how to autograde these kinds of assignments

Thanks to our fantastic tech team (Jason mainly, with Paul supporting some new tools) and UIT we’ve got Virtual Programming Lab running on eClass.  It provides us with a way of creating interactive programming assignments with direct integration with the Moodle system in eClass… including directly into the grade book.  I’ve been testing out a number of different language including:

  1. Verilog https://youtu.be/orVWWmsYSlA
  2. Python https://youtu.be/CD0P5G18mtU
  3. Java https://youtu.be/Nk_093HWkHM
  4. C  https://youtu.be/MxIM5wE2CVw
  5. C++ https://youtu.be/TWqFol8ek_Q
  6. Bash https://youtu.be/g4OWn9wQHcU
  7. Go  https://youtu.be/BtWs243Pmy8 
  8. Haskell https://youtu.be/57dcszPFyEU
  9. Ada https://youtu.be/L8rqBlcX0To
  10. Fortran  https://youtu.be/1pSqLyLSHI8
  11. Matlab https://youtu.be/NNZqkF07Qlk
The same “Hello World” VPL assignment but, this time, with a regular expression in the test that searches for any instance of the string “World!”.

Each of these has a short video available to show you an auto-graded “hello world” example in these languages.  Some of these are directly supported by our local VPL server, while others were tested using the regular VPL server in Spain.  Local support is possible if there is a need and it is possible to extend VPL to support other languages than those mentioned.

Other languages are possible. Assembler for x86 and MIPS, R, Kotlin, Scala, Erlang, Pascal, etc. The point here isn’t to compare the merits of individual computer languages but, rather, to illustrate the versatility of VPL to be integrated in a variety of courses with programming elements. Down the road, I’m looking forward to seeing if we can use it with Maple and chip simulator packages like MPLAB X. Stay tuned…

In the examples above we verify the correctness of the student’s “Hello World” string by comparing it to another string. But it’s also possible to do more complex testing using dreaded (by me, at least) regular expressions. RegEx tests can be done in the “vpl_evaluate.cases” file so that VPL can, for instance, look for a particular key word.

Example Scripts for Running and Evaluating

While you can develop VPL activities with little to no customization, sometimes it’s important to do so. A number of people have developed run and evaluation scripts (vpl_run.sh and vpl_evaluate.sh) that can serve as starting points for doing your own custom work.

Sending Grade and Comments to Moodle

In order for students to receive grades and comments you need to have some method of outputting two strings during the evaluation process:

  • “Grade :=>> “
  • “Comment :=>> “

Where the Grade string typically ends with a number and the Comment string ends with more text. This is typically done via the vpl_evaluate.sh shell script and is illustrated in the 2015 paper by Thibaut.

Here’s the simplest example that I could think of to show how the grading can work. Here are the steps.

  1. Create a new VPL assignment
  2. Set the location of the VPL server
  3. Add a “requested file” and put a little comment in it. Nothing more. Call it nothingHere.txt or something like that.
  4. Fill in vpl_run.sh as shown below.
  5. Fill in vpl_evaluate.sh as shown below.
  6. Test it by logging in as a “demo student” and verifying by editing the trivial nothingHere.txt, hitting “run” and “evaluate” icons.
Simplest example of assigning a grade to a student. These are the two main script files. The student will receive a 100% grade by effectively submitting nothing. When both scripts run they create a file called “vpl_execution”. That’s what gets executed by VPL, so controlling the contents of vpl_execution is key.

Note this comment from the developer: “The default vpl_evaluate.sh uses the result of vpl_run.sh (vpl_execute) as the program to be evaluated.
Notice that the output of the execution of vpl_execute, generated by vpl_evaluate.sh, is the result of the evaluation. You can see the format of the output of this execution in the documentation.”

The interaction between vpl_run.sh and vpl_evaluate.sh via vpl_execute are complex and non-intuitive. Dominique Thiebaut documented a model solution for non-standard VPL activities here after a discussion on the Moodle forum.

There’s nothing in vpl_debug. Nothing in vpl_evaluate.cases. It’s all handled by vpl_run.sh and vpl_evaluate.sh. This is pretty much how Thibaut does it in most of his examples. So, if you’re going off the beaten track and your assignments aren’t covered by the default approach, this is one way to go.

More Complex Examples

Unit Testing

jUnit and Java programs

[this is covered in another blog post… need link]

another option: https://github.com/bytebang/vpl-junit

Unity and C programs.

My first attempt at using a unit test framework for C programs in VPL uses the Unity framework. The reason is simple: Unity only requires that a few files be added to a C project. It also has been tested on a number of embedded build systems, including MPLAB X. The Unity unit test will output results of the unit test to the standard output, like

printf() does. The output will state whether the test was successful. So in vpl_evaluate.cases we’re going to search for the string “0 Failures” using the regular expression

/(0\sFailures)/i

The regEx that will search for “0 Failures” in the output of the Evaluation test.

It’s not very sophisticated and could be made more robust, but it’s sufficient to get started. I’ll insert it into vpl_evaluate.cases like this, via the Test Cases menu item:

using the regular expression /(0\sFailures)/i to look up the Unity result saying that all tests passed in order to auto-grade students.

Here are the other things that need to be added to the Gear -> Execution files:

  1. TestDumbExample.c (modified version from ThrowTheSwitch’s Unity page)
  2. TestDumbExample.h (modified version from ThrowTheSwitch’s Unity page)
  3. DumbExample.h (modified version from ThrowTheSwitch’s Unity page)
  4. unity.h (via ThrowTheSwitch’s Github)
  5. unity.c (via ThrowTheSwitch’s Github)
  6. unity_internals.h (via ThrowTheSwitch’s Github)

And it should look like this:

The new files in the Execution Files list. These include specific files for testing (TestDumbExample.c, DumbExample.h) of a particular student’s code submission (DumbExample.c, found in the Requested Files section). The other files, unity.h, unity.c and unity_internals.h, are the core unit testing system.

Next, add a file as a student submission template. This is placed in Gear -> Requested Files. Call it DumbExample.c:

This is the code under test.

Extra Resources

James Andrew Smith is an associate professor in Electrical Engineering and Computer Science Department in York University’s Lassonde School.  He lived in Strasbourg, France and taught at the INSA Strasbourg and Hochschule Karlsruhe while on sabbatical in 2018-19 with his wife and kids.