VPL: A Java Unit Test Example

The following is an example of the files needed to create a working Virtual Programming Lab exercise for Java:

• vpl_run.sh
• vpl_evaluate.sh
• MainClass.java
• StudentSolution.java
• TeacherReferenceSolutions.java
• TheTestClass.java

With this you can evaluate one student submission against a single reference solution. Two methods are run: one is the student’s method and one is the teacher’s method. The output of both will be compared using jUnit.

Here are the two VPL scripts, vpl_run.sh and vpl_evaulate.sh:

vpl_run.sh

(for letting students practice)

#!/bin/bash
#load common script and check programs
# vpl_run_JDK17_Works_Jan2023.sh
# updated on Jan 22, 2024 for a new jUnit file called TheTestClass.java

. common_script.sh
check_program javac
check_program java
get_source_files java

#compile all .java files

# updated Jan 2023:
# tcsh (on indigo)
# setenv CLASSPATH /eecs/local/pkg/junit/junit.jar
# bash (on VPL)
export CLASSPATH=$CLASSPATH:/eecs/local/pkg/junit/junit.jar
export CLASSPATH=$CLASSPATH:/eecs/local/pkg/junit/hamcrest-core.jar

#javac *.java
javac -J-Xmx16m -Xlint:deprecation *.java

if [ "$?" -ne "0" ] ; then

  echo "Not compiled"
  exit 0

fi


cat common_script.sh > vpl_execution
# updated Jan 2023:
# Version 1 (run): Just does the unit test... bypasses main.  doesn't pretend to assign grade.
echo "java -enableassertions -cp $CLASSPATH:/eecs/local/pkg/junit/junit.jar org.junit.runner.JUnitCore TheTestClass" >> vpl_execution
# Version 2 (evaluate):  (Runs Main and shows comment and score... could be confusing to students.  Don't use when running.  Use during evaluate.)
#echo "/eecs/local/pkg/jdk17/bin/java -enableassertions -cp $CLASSPATH:/eecs/local/pkg/junit/junit.jar Main" >> vpl_execution

chmod +x vpl_execution

vpl_evaluate.sh

(for submitted grades to eClass)

#!/bin/bash
#load common script and check programs
# vpl_evaluate_JDK17_Works_Jan2023.sh
# updated Jan 22, 2024 for Main class called MainClass

. common_script.sh
check_program javac
check_program java
get_source_files java

#compile all .java files

# updated Jan 2023:
# tcsh (on indigo)
# setenv CLASSPATH /eecs/local/pkg/junit/junit.jar
# bash (on VPL)
export CLASSPATH=$CLASSPATH:/eecs/local/pkg/junit/junit.jar
export CLASSPATH=$CLASSPATH:/eecs/local/pkg/junit/hamcrest-core.jar

javac -J-Xmx16m -Xlint:deprecation *.java

if [ "$?" -ne "0" ] ; then

  echo "Not compiled"
  exit 0

fi

cat common_script.sh > vpl_execution
# updated Jan 2023:
# Version 1 (run): Just does the unit test... bypasses main.  doesn't pretend to assign grade.
# echo "java -enableassertions -cp $CLASSPATH:/eecs/local/pkg/junit/junit.jar org.junit.runner.JUnitCore MyTest" >> vpl_execution
# Version 2 (evaluate):  (Runs Main and shows comment and score... could be confusing to students.  Don't use when running.  Use during evaluate.)
echo "/eecs/local/pkg/jdk17/bin/java -enableassertions -cp $CLASSPATH:/eecs/local/pkg/junit/junit.jar MainClass" >> vpl_execution


chmod +x vpl_execution

MainClass.java

(in the execution files section)

/**
 * An example that demonstrates running and post-processing JUnit tests in order to grade a VPL activity.
 */

import java.util.Formatter;
import java.util.Locale;

public class MainClass {

    /**
     * Run some unit tests and process the results, calculating an overall grade for this activity.
     * Print out comments for each test and the overall grade, using the format required by vpl_execution.
     * @param args
     */
    public static void main(String[] args) {
        TheTestClass t = new TheTestClass();
        int grade = 0;
        try {
            t.testOne(); //worth 1 mark
            System.out.println(formatOutput("Test One", "1", null));
            grade += 1;     // Test is worth 1 mark.  Update grade.
        } catch (AssertionError e) {
            System.out.println(formatOutput("Test One", "1", e));
        }
        System.out.println("Grade :=>> "+grade);
    }

    /**
     * Format one or more comments for a test.
     * @param testName
     * @param value
     * @param e
     * @return
     */
    private static String formatOutput(String testName, String value, AssertionError e) {
        StringBuilder sb = new StringBuilder();
        Formatter f = new Formatter(sb, Locale.getDefault());
        String grade = (e == null ? value : "0");
        f.format("Comment :=>> %s: %s. %s marks\n", testName, (e == null ? "success" : "failure"), grade);
        if (e != null) {
            f.format("<|-- \n%s\n --|>\n", e.getMessage());
        }
        return sb.toString();
    }
}

StudentSolutions.java

(in the requested files section)

This is the fake student’s attempt at solving the problem.

/* StudentSolution.java
 * An example of a file that a student might submit */
public class StudentSolution {

    public Integer studentMethod(Integer A, Integer B) {

        Integer C = 0;

        C = A + B;

        return C;
    }
}

TeacherReferenceSolutions.java

(in the execution files section)

This is the teacher reference solution that the student solution is compared two. Both receive the same inputs.

/* TeacherReferenceSolutions.java.
 * this is where the "answer" is found. */
public class TeacherReferenceSolutions {


    /* Method 1 for the Teacher (it's the answer) */
    public static Integer teacherMethod1(Integer A, Integer B) {

        Integer C = 0;

        C = A + B;

        return C;
    }


}

TheTestClass.java

(in the execution files section)

This is the file that conducts the comparison between the student and teacher methods. It runs 20 pairs of integers against the student method to see if the outputs match up.

import org.junit.*;
import org.junit.Assert;
import org.junit.Test;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.function.BiFunction;


public class TheTestClass {


    @Test
    // method
    public void testOne() {

        TeacherReferenceSolutions theReference = new TeacherReferenceSolutions();
        StudentSolution theStudentAttempt = new StudentSolution();
        Integer studentAnswer = null;
        Integer teacherAnswer = null;
        ArrayList<Integer> inputOne = new ArrayList<>();
        ArrayList<Integer> inputTwo = new ArrayList<>();
        Integer minInput = -100;            // Boundaries on inputs (max, min)
        Integer maxInput = +400;
        final Integer TESTTOTALNUMBER = 20;  // Twenty sub-tests run on student vs. teacher.

        /* build up a unique set of test inputs */
        for (int i = 0; i < TESTTOTALNUMBER; i++) {
            inputOne.add(new Random().nextInt(maxInput - minInput) + minInput);
            inputTwo.add(new Random().nextInt(maxInput - minInput) + minInput);
        }


        // First loop.  Print to screen messages that students can more easily decode.
        for (Integer testIteration = 0; testIteration < TESTTOTALNUMBER; testIteration++) {

            Integer firstInput = inputOne.get(testIteration);
            Integer secondInput = inputTwo.get(testIteration);

            /* Run the teacher solution once.  Get the reference output. */
            teacherAnswer = TeacherReferenceSolutions.teacherMethod1(firstInput,secondInput);
            System.out.println("Subtest " + testIteration + " : testing against inputs : " + firstInput + " and " + secondInput);

            // Run the student's answer.  Get the student's output. */
            studentAnswer = theStudentAttempt.studentMethod(firstInput,secondInput);


            try {
                Assert.assertEquals(teacherAnswer, studentAnswer);
                System.out.println("Sub-Test " + testIteration + " passed.");
                System.out.println("Teacher solution: " + teacherAnswer + " ... versus student solution: " + studentAnswer);
                System.out.println("------------------------------------------------------");
                System.out.println("");

            } catch (AssertionError e) {
                System.out.println("");
                System.out.println("****************************************************");
                System.out.println("vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv");
                System.out.println("\n\t\t\t Sub-Test " + testIteration + " + DID NOT PASS!! \t\t\t\t");
                System.out.println("For inputs " + inputOne.get(testIteration) + " and " + inputTwo.get(testIteration) + " we got the following output:");
                System.out.println(e.getMessage());
                System.out.println("");
                System.out.println("^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^");
                System.out.println("****************************************************");
                System.out.println("");
            }
        }


        // Second loop.  Allows Junit to register the error in VPL or in the IDE.
        for (Integer testIteration = 0; testIteration < TESTTOTALNUMBER; testIteration++) {

            Integer firstInput = inputOne.get(testIteration);
            Integer secondInput = inputTwo.get(testIteration);

            /* Run the teacher solution once.  Get the reference output. */
            teacherAnswer = TeacherReferenceSolutions.teacherMethod1(firstInput,secondInput);
            System.out.println("Subtest " + testIteration + " : testing against inputs : " + inputOne.get(testIteration) + " and " + inputTwo.get(testIteration));
            // Run the student's answer.  Get the student's output. */
            studentAnswer = theStudentAttempt.studentMethod(firstInput,secondInput);

            // no try-catch here.  This will register with the IDE or VPL
            Assert.assertEquals(teacherAnswer, studentAnswer);


        }

    }  // end method




}

---

James Andrew Smith is a Professional Engineer and Associate Professor in the Electrical Engineering and Computer Science Department of York University’s Lassonde School, with degrees in Electrical and Mechanical Engineering from the University of Alberta and McGill University.  Previously a program director in biomedical engineering, his research background spans robotics, locomotion, human birth and engineering education. While on sabbatical in 2018-19 with his wife and kids he lived in Strasbourg, France and he taught at the INSA Strasbourg and Hochschule Karlsruhe and wrote about his personal and professional perspectives.  James is a proponent of using social media to advocate for justice, equity, diversity and inclusion as well as evidence-based applications of research in the public sphere. You can find him on Twitter. Originally from Québec City, he now lives in Toronto, Canada.