NB This post is a work in progress.

This post documents the process I used to programmatically generate multiple variations on a quiz question for the Brightspace virtual learning environment. The quiz question displays an example C program and asks what number it will print out. In each generated variation of the question, placeholder variable names (VAR1, VAR2, VAR3) and numerical values (VAL1, VAL2, VAL3) are replaced by different strings.

The quiz feature in Brightspace provides an “Arithmetic” question type that allows certain values in the question to be randomised and the correct answer to be calculated with a user-specified formula. Unfortunately, the set of supported mathematical operators is very limited. In my code analysis questions, changing the numerical values in the code can alter the flow of control through the program. This forces students to trace the flow of control very carefully, even when attempting the quiz multiple times. Calculating the value printed by each version of the program is more complex than what Brightspace’s Arithmetic question type supports.

My alternative approach for programmatically generating many variations of a question that use the same code structure:

Example template code (“code.c”) for a set of variations on the same code analysis question:

Shown below is an example PNG image produced from one variation on the code template above. “VAR1” has been replaced by “alice”, “VAL1” has been replaced by “-1”, and so on. The modified code has been automatically rendered as a PNG image using ImageMagick. Students are asked to identify the number that the program prints out.

By automating the process of generating variations on a single code template, it’s possible to create code analysis quizzes that remain challenging to students even when attempted multiple times.

On my computer (running Xubuntu Linux 20.04), using ImageMagick to generate a PNG image from each file of example C code required a change to its default security policy. As root, I commented out the following line in `/etc/ImageMagick-6/policy.xml`

The following Python program is used to generate a question list file, with a line of text describing each variation on the question.

Shown below is an example of the “question_list.csv” file produced by the above Python program. With the exception of the first two lines, each line defines a different variation on the question. The first three items on each line are the strings to substitute for VAR1, VAR2 and VAR3. The last three items on each line are the strings to substitute for VAL1, VAL2 and VAL3. Different text markers (as many or few as required) can be used by editing line 2.

Once the file “question_list.csv” is generated, we’re ready to generate the code PNG images and the CSV file containing the information for each question which will be used to import the question variations into Brightspace.

Shown below is an example of the Brightspace CSV import file, “brightspace_import_file.csv”. This is generated by the program above.

NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-001,,,
Title,COMP2109-FlowOfControl-If1-001,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-001.png,,,
InputBox,1,6,,
Answer,100,6,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-002,,,
Title,COMP2109-FlowOfControl-If1-002,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-002.png,,,
InputBox,1,6,,
Answer,100,6,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-003,,,
Title,COMP2109-FlowOfControl-If1-003,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-003.png,,,
InputBox,1,6,,
Answer,100,1,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-004,,,
Title,COMP2109-FlowOfControl-If1-004,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-004.png,,,
InputBox,1,6,,
Answer,100,-6,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-005,,,
Title,COMP2109-FlowOfControl-If1-005,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-005.png,,,
InputBox,1,6,,
Answer,100,3,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-006,,,
Title,COMP2109-FlowOfControl-If1-006,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-006.png,,,
InputBox,1,6,,
Answer,100,-9,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-007,,,
Title,COMP2109-FlowOfControl-If1-007,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-007.png,,,
InputBox,1,6,,
Answer,100,-8,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-008,,,
Title,COMP2109-FlowOfControl-If1-008,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-008.png,,,
InputBox,1,6,,
Answer,100,-5,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-009,,,
Title,COMP2109-FlowOfControl-If1-009,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-009.png,,,
InputBox,1,6,,
Answer,100,2,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-010,,,
Title,COMP2109-FlowOfControl-If1-010,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-010.png,,,
InputBox,1,6,,
Answer,100,2,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-011,,,
Title,COMP2109-FlowOfControl-If1-011,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-011.png,,,
InputBox,1,6,,
Answer,100,-1,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-012,,,
Title,COMP2109-FlowOfControl-If1-012,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-012.png,,,
InputBox,1,6,,
Answer,100,-1,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-013,,,
Title,COMP2109-FlowOfControl-If1-013,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-013.png,,,
InputBox,1,6,,
Answer,100,6,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-014,,,
Title,COMP2109-FlowOfControl-If1-014,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-014.png,,,
InputBox,1,6,,
Answer,100,4,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-015,,,
Title,COMP2109-FlowOfControl-If1-015,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-015.png,,,
InputBox,1,6,,
Answer,100,4,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-016,,,
Title,COMP2109-FlowOfControl-If1-016,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-016.png,,,
InputBox,1,6,,
Answer,100,3,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-017,,,
Title,COMP2109-FlowOfControl-If1-017,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-017.png,,,
InputBox,1,6,,
Answer,100,9,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-018,,,
Title,COMP2109-FlowOfControl-If1-018,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-018.png,,,
InputBox,1,6,,
Answer,100,-10,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-019,,,
Title,COMP2109-FlowOfControl-If1-019,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-019.png,,,
InputBox,1,6,,
Answer,100,-6,,
Hint,,,,
Feedback,,,,
,,,,
NewQuestion,SA,,,
ID,COMP2109-FlowOfControl-If1-020,,,
Title,COMP2109-FlowOfControl-If1-020,,,
QuestionText,What number will this program print?,,,
Points,1,,,
Difficulty,1,,,
Image,images/COMP2109-FlowOfControl-If1/COMP2109-FlowOfControl-If1-020.png,,,
InputBox,1,6,,
Answer,100,-4,,
Hint,,,,
Feedback,,,,
,,,,