The assessment of student learning in the formative stage of a university programming class has become very challenging for a number of reasons:
- the pandemic has generally disrupted learning processes at the elementary and high school levels
- Prior to the pandemic, reading and note-taking habits had changed, possibly making them less effective.
- IDE-specific tools catch errors and prompt students to “tab through” to a correct solution
- Online communities on Discord and commercial pirates like Course Hero
- External tools like ChatGPT and GitHub Copilot provide convenient, just-correct-enough solutions to shortcut learning during unsupervised formative exercises.
Here I focus on developing a scalable series of programming exercises, designed to be completed on the computer, either in an unsupervised or in a supervised context. With the right support systems these exercises could form a bank of questions to be used in a testing environment.
One of the key elements in these exercises is the use of visualization tools like flowcharts. By introducing students to flowcharts as part of the formative assessment process the hope is that their usefulness will be effectively communicated to students so that they, also, adopt them as tools for the development of procedural programs.
To that end we use Diagon, a utility that converts pseudo-code into ASCII-art flowcharts or equations. The advantage of using ASCII art is that the text can be embedded in standard text files and processed in a straight-forward manner.
The workflow is this:
- Write a program with while loops and if statements in Matlab (for loops are not supported in Diagon)
- Copy and paste from Matlab to Diagon and…
- Add quaotation marks within the parentheses found beside the if and while keywords
- Add quotation marks to stand-alone statements
- Add curly braces and remove END keywords
- Convert elseif to else if
- Add START and END points
- Run the resulting script through the Diagon website or command line program
The file creation process is summarized in the following diagram:
In the image above, we’re aiming to have students create a combination of loops and conditional statements which vary based on two input variables (s and t) and output a single numeric result (a).
Given sufficient time I might be able to come up with a text processing program to automate the conversion. I’ll park that idea for another time.
Now the idea is that we can generate an arbitrary number of variations of these questions. Here, I’ve created four variations and stored them in three different files:
- matlab_scripts.txt
- the reference code (teacher solution)
- diagon_scripts.txt
- the pseudo-code interpretation of the Matlab code
- diagon_flowcharts.txt
- the flowchart interpretation of the pseudo-code
Here is the first file, matlab_scripts.txt:
% matlab_scripts.txt
% function a = example_flow_chart_function(s,t)
% first
% ----- start ----
a = 0;
i=1;
while(i<10)
if(t < 10)
a = a+2;
elseif(t<20)
a = a+4;
else
a = a+5;
end
i = i+1;
end
% ----- end ----
% second
% ----- start ----
a = 0;
i=1;
while(i<5)
if(t < 2)
a = a+2;
elseif(t<4)
a = a+2;
else
a = a+1;
end
i = i+1;
end
% ----- end ----
% third
% ----- start ----
a = 0;
i=1;
while(i<10)
if(t > 5)
a = a+1;
elseif(t>2)
a = a+3;
else
a = a+10;
end
i = i+1;
end
% ----- end ----
% fourth
% ----- start ----
a = 0;
i=1;
while(i<=10)
if(t >= 5)
a = a+1;
elseif(t>=2)
a = a+3;
else
a = a+10;
end
i = i+1;
end
% ----- end ----Here is the second file, diagon_scripts.txt
% diagon_scripts.txt
% first
% ----- start ----
"START";
"a = 0";
"i=1";
while("i<10")
{
if("t < 10")
{
"a = a+2";
}
else if("t<20")
{
"a = a+4";
}
else
{
"a = a+5";
}
"i = i+1";
}
"END"
% ----- end ----
% second
% ----- start ----
"START";
"a = 0";
"i=1";
while("i<5")
{
if("t < 2"){
"a = a+2";}
else if("t<4"){
"a = a+2";
}
else{
"a = a+1";
}
"i = i+1";
}
"END"
% ----- end ----
% third
% ----- start ----
"START";
"a = 0";
"i=1";
while("i<10")
{
if("t > 5"){
"a = a+1";}
else if("t>2"){
"a = a+3";}
else{
"a = a+10";
}
"i = i+1";
}
"END"
% ----- end ----
% fourth
% ----- start ----
"START";
"a = 0";
"i=1";
while("i <= 10")
{
if("t >= 5"){
"a = a+1";}
else if("t >= 2"){
"a = a+3";}
else{
"a = a+10";
}
"i = i+1";
}
"END"
% ----- end ----Here is the third file, daigon_flowcharts.txt
% diagon_flowcharts.txt
% first
% ----- start ----
┌─────┐
│START│
└──┬──┘
┌──▽──┐
│a = 0│
└──┬──┘
┌─▽─┐
│i=1│
└─┬─┘
▽___
╱ ╲
____╱ i<10 ╲______________________
│ no ╲ ╱ │
│ ╲____╱ │
│ │yes │
│ __▽___ │
│ ╱ ╲ ┌───────┐ │
│ ╱ t < 10 ╲___________│a = a+2│ │
│ ╲ ╱yes └───┬───┘ │
│ ╲______╱ │ │
│ │no │ │
│ _▽__ │ │
│ ╱ ╲ ┌───────┐ │ │
│ ╱ t<20 ╲___│a = a+4│ │ │
│ ╲ ╱yes└───┬───┘ │ │
│ ╲____╱ │ │ │
│ │no │ │ │
│ ┌───▽───┐ │ │ │
│ │a = a+5│ │ │ │
│ └───┬───┘ │ │ │
│ └─────┬─────┴────────┘ │
│ ┌───▽───┐ │
│ │i = i+1│ │
│ └───┬───┘ │
│ └────────────────────┘
┌─▽─┐
│END│
└───┘
% ----- end ----
% second
% ----- start ----
┌─────┐
│START│
└──┬──┘
┌──▽──┐
│a = 0│
└──┬──┘
┌─▽─┐
│i=1│
└─┬─┘
▽__
╱ ╲
____╱ i<5 ╲______________________
│ no ╲ ╱ │
│ ╲___╱ │
│ │yes │
│ __▽__ │
│ ╱ ╲ ┌───────┐ │
│ ╱ t < 2 ╲___________│a = a+2│ │
│ ╲ ╱yes └───┬───┘ │
│ ╲_____╱ │ │
│ │no │ │
│ _▽_ │ │
│ ╱ ╲ ┌───────┐ │ │
│ ╱ t<4 ╲___│a = a+2│ │ │
│ ╲ ╱yes└───┬───┘ │ │
│ ╲___╱ │ │ │
│ │no │ │ │
│ ┌───▽───┐ │ │ │
│ │a = a+1│ │ │ │
│ └───┬───┘ │ │ │
│ └────┬─────┴────────┘ │
│ ┌───▽───┐ │
│ │i = i+1│ │
│ └───┬───┘ │
│ └────────────────────┘
┌─▽─┐
│END│
└───┘
% ----- end ----
% third
% ----- start ----
┌─────┐
│START│
└──┬──┘
┌──▽──┐
│a = 0│
└──┬──┘
┌─▽─┐
│i=1│
└─┬─┘
▽___
╱ ╲
____╱ i<10 ╲_____________________
│ no ╲ ╱ │
│ ╲____╱ │
│ │yes │
│ __▽__ │
│ ╱ ╲ ┌───────┐ │
│ ╱ t > 5 ╲___________│a = a+1│ │
│ ╲ ╱yes └───┬───┘ │
│ ╲_____╱ │ │
│ │no │ │
│ _▽_ │ │
│ ╱ ╲ ┌───────┐ │ │
│ ╱ t>2 ╲___│a = a+3│ │ │
│ ╲ ╱yes└───┬───┘ │ │
│ ╲___╱ │ │ │
│ │no │ │ │
│ ┌────▽───┐ │ │ │
│ │a = a+10│ │ │ │
│ └────┬───┘ │ │ │
│ └────┬─────┴────────┘ │
│ ┌───▽───┐ │
│ │i = i+1│ │
│ └───┬───┘ │
│ └────────────────────┘
┌─▽─┐
│END│
└───┘
% ----- end ----
% fourth
% ----- start ----
┌─────┐
│START│
└──┬──┘
┌──▽──┐
│a = 0│
└──┬──┘
┌─▽─┐
│i=1│
└─┬─┘
__▽____
╱ ╲
____╱ i <= 10 ╲______________________
│ no ╲ ╱ │
│ ╲_______╱ │
│ │yes │
│ __▽___ │
│ ╱ ╲ ┌───────┐ │
│ ╱ t >= 5 ╲____________│a = a+1│ │
│ ╲ ╱yes └───┬───┘ │
│ ╲______╱ │ │
│ │no │ │
│ __▽___ │ │
│ ╱ ╲ ┌───────┐ │ │
│ ╱ t >= 2 ╲___│a = a+3│ │ │
│ ╲ ╱yes└───┬───┘ │ │
│ ╲______╱ │ │ │
│ │no │ │ │
│ ┌────▽───┐ │ │ │
│ │a = a+10│ │ │ │
│ └────┬───┘ │ │ │
│ └─────┬──────┴────────┘ │
│ ┌───▽───┐ │
│ │i = i+1│ │
│ └───┬───┘ │
│ └─────────────────────┘
┌─▽─┐
│END│
└───┘
% ----- end ----
To start the question for the student, we run the example_flowchart_question() function. It’s stored in a file called example_flowchart_question.m:
% example_flowchart_question.m
function example_flowchart_question()
% update this for flowcharts...
DELAYTIME = 0.2; % [seconds]
% randomize selection.
% could also do something like add up all digits in the student ID and
% choose the least significant digit of the result and divide by 10.
whichexample = rand(1);
disp("Dear student: write a function that implements the following flowchart.")
disp("There are two input variables (s and t) and one returned variable, a.")
disp("Your program should begin with the line")
disp(" ")
disp("function a = studentfunction(s,t)")
disp(" ")
disp("and it should end with the keyword ")
disp(" ")
disp("end")
disp(" ")
disp("on the last line.")
disp(" ")
pause(DELAYTIME);
disp("Your function's text will be automatically graded. You can only do this once.")
disp("So verify your answer before submitting it.")
disp(" ")
disp("Here is the flowchart that you are to implement in Matlab:")
disp(" ")
mystringcode = find_specific_question(whichexample); % select at random.
myasciiart = findasciiart(whichexample);
disp(" ")
pause(DELAYTIME); pause(DELAYTIME); pause(DELAYTIME); pause(DELAYTIME);
disp("Pretending to wait for student to write their file...")
pause(DELAYTIME); pause(DELAYTIME); pause(DELAYTIME); pause(DELAYTIME);
disp("Generating the teacher solution...")
pause(DELAYTIME); pause(DELAYTIME); pause(DELAYTIME);pause(DELAYTIME);
% take the array of strings and write a file
fileID = fopen('teacherfunction.m','w');
fprintf(fileID,'function a = teacherfunction(s,t)\n'); % opening line.
for i = 1: numel(mystringcode)
fprintf(fileID,'%s\n',mystringcode(i));
end
fclose(fileID);
disp("reading the student solution...")
pause(DELAYTIME); pause(DELAYTIME); pause(DELAYTIME);pause(DELAYTIME);
disp("comparing student and teacher solutions...")
% Test the myFunction with sample input
input1 = randi([0,100]);
input2 = randi([0,100]);
expectedOutput = teacherfunction(input1,input2); % teacher's solution (reference)
testedOutput = studentfunction(input1,input2); % student's solution.
try
assert( testedOutput == expectedOutput);
result = true;
catch
result = false;
end
if (result == true)
disp([sprintf("Test succeeded. The tested function worked as tested.")]);
disp([sprintf("Inputs %d and %d (s and t) yielded an answer of %d for an expected answer of %d", input1, input2, testedOutput, expectedOutput)])
else
disp([sprintf("Test failed. The tested function did not work as tested. ")]);
disp([sprintf("Inputs %d and %d (s and t) yielded an answer of %d for an expected answer of %d", input1, input2, testedOutput, expectedOutput)])
end
endThe above function calls find_specific_question.m in order to provide the reference solution (e.g. teacher solution) to the Matlab script:
% find_specific_question.m
function whichone = find_specific_question(selection)
filename = 'matlab_scripts.txt'; % this is the code, not the ASCII art.
whichone = "-1";
% selection is from 0 to 1.
if((selection>0) && (selection <=1))
% scan the file with questions in it and determine how many we have.
if (exist(filename, 'file') == 2)
% File exists
% read the file to obtain the equation
% default to another equation if no file is present.
fileID = fopen(filename,'r');
firstLine = fgets(fileID);
fclose(fileID);
% disp("DEBUG: reading file");
lines = readlines(filename); % read the file into array.
str1 = "% ----- start ----";
str2 = "% ----- end ----";
ind_log_start = strcmp(lines,str1); % index all the starts
index_start = find(ind_log_start);
ind_log_end = strcmp(lines,str2); % index all the ends
index_end = find(ind_log_end);
num_eqtns = length(index_start); % how many equations?
% find the code desired.
% desiredvalue = lines((index_start(ceil(selection*num_eqtns)))+1);
% go from (index_start(ceil(selection*num_eqtns)))+1
% to
% (index_end(ceil(selection*num_eqtns)))-1
length_of_chart = ...
((index_end(ceil(selection*num_eqtns)))-1) - ...
(index_start(ceil(selection*num_eqtns)))+1;
for(i=1:1:(length_of_chart-1))
%disp(lines(i+index_start(ceil(selection*num_eqtns))));
desiredvalue(i)= lines(i+index_start(ceil(selection*num_eqtns)));
end
whichone = desiredvalue;
else
% File does not exist
disp("File error. Report this to instructor.")
whichone = "0";
end
else
disp("wrong selection made. Should be a value above zero and less than 1.")
endand then for extracting the corresponding flowchart using the same index but in the “flowcharts” file and to be able to display the flowchart to the student we have findasciiart.m:
% findasciiart.m
% Prompt student
function CellArray = findasciiart(selection)
filename = 'diagon_flowcharts.txt';
CellArray = cell(1,1);
[CellArray{:}]=deal(NaN);
% selection is from 0 to 1.
if((selection>0) && (selection <=1))
% scan the file with questions in it and determine how many we have.
if exist(filename, 'file') == 2
% File exists
% read the file to obtain the equation
% default to another equation if no file is present.
fileID = fopen(filename,'r');
firstLine = fgets(fileID);
fclose(fileID);
lines = readlines(filename) ; % read the file into array.
str1 = "% ----- start ----";
str2 = "% ----- end ----";
ind_log_start = strcmp(lines,str1); % index all the starts
index_start = find(ind_log_start);
ind_log_end = strcmp(lines,str2); % index all the ends
index_end = find(ind_log_end);
num_eqtns = length(index_start); % how many equations?
% find the equation desired.
% grab all the strings until you hit str2.
first_index = (index_start(ceil(selection*num_eqtns)))+1;
last_index = (index_end(ceil(selection*num_eqtns)))-1;
linenumber = 1;
for(counter = first_index:1:last_index)
mything(linenumber) = lines(counter);
linenumber = linenumber +1;
end
whichone = mything;
% convert to cell array
CellArray = cellstr(mything);
%disp("start the print")
% print the flowchart to the screen
for i = 1:numel(CellArray)
fprintf('%s\n',CellArray{i});
end
else
% File does not exist
disp("File error. Report this to instructor.")
CellArray = "0";
end
%disp("end the print")
else
disp("wrong selection made. Should be a value above zero and less than 1.")
endprocessing the matlab_scripts.txt file, a new file will be created. The
The student will be provided with a template that looks like this, with the first and last lines completed:
% studentfunction.m (template)
function a = studentfunction(s,t)
endand then the student will write their solution:
% studentfunction.m. (completed)
function a = studentfunction(s,t)
% student writes a program in here that implements the flow chart.
a = 0;
i=1;
while(i<10)
if(t < 10)
a = a+2;
elseif(t<20)
a = a+4;
else
a = a+5;
end
i = i+1;
end
endWhat next? Copy each of the files above into a file folder in Matlab. Include the fake student solution, “studentfunction.m” file, shown above, too. Then, in the command window type in:
>> example_flowchart_question
and then the program will run and compare your “student solution” to one of the flowcharts that it chooses at random. Most of the time you should get a reply saying that the solution is wrong, like this:
Test failed. The tested function did not work as tested.
Inputs 96 and 97 (s and t) yielded an answer of 45 for an expected answer of 9
So, run the program a few more times and, eventually, you’ll land on the right one:
Test succeeded. The tested function worked as tested.
Inputs 98 and 96 (s and t) yielded an answer of 45 for an expected answer of 45
What’s next? This needs to be integrated into VPL, web submit, lab test mode, or whatever your preferred assignment system is. (me? I’ll be working on integrating it into web submit later)
James Andrew Smith is a Professional Engineer and Associate Professor in the Electrical Engineering and Computer Science Department of York University’s Lassonde School, with degrees in Electrical and Mechanical Engineering from the University of Alberta and McGill University. Previously a program director in biomedical engineering, his research background spans robotics, locomotion, human birth and engineering education. While on sabbatical in 2018-19 with his wife and kids he lived in Strasbourg, France and he taught at the INSA Strasbourg and Hochschule Karlsruhe and wrote about his personal and professional perspectives. James is a proponent of using social media to advocate for justice, equity, diversity and inclusion as well as evidence-based applications of research in the public sphere. You can find him on Twitter. Originally from Québec City, he now lives in Toronto, Canada.