ASSESSMENT OF MULTIPLE CHOICE TASKS

1. Assessment options in ONYX
2. Implementation of the Model Regulations on Conducting and Evaluating Multiple Choice Exams
  1. Multiple-choice questions
  2. Multiple-response questions

ONYX allows three different ways to award points for automatically graded tasks. Depending on the type of task, some options may not be available. The tables below contain sample evaluations for multiple-choice and multiple-response tasks. The number of (correct) answer options and the amount of points may differ depending on your questions. The method for entering points in ONYX is similar to what is shown in the “Points” columns, even if the values are not exactly the same. Please also see the ONYX handbook for specific examples of how points are awarded depending on students’ selections.

The probability of passing a multiple-choice test by guessing varies depending on the scoring method and the number of (correct) answers. We have therefore included brief instructions on how to comply with the Model Regulations on Conducting and Evaluating Multiple Choice Exams using ONYX.

Assessment options in ONYX

In general:

The lowest possible score is 0.
If no selection is made, 0 points are given.

a) All correct answers

Points are only awarded if every correct answer is selected. Participants therefore receive either the maximum amount of points or 0 points.

#	Question/ task	Answers	Points
1	Select the correct response(s)!	Correct option 1 Incorrect option 1 Correct option 2 Incorrect option 2	2
2	Select the correct response(s)!	Correct option 1 Correct option 2 Correct option 3 Correct option 4	2

b) Points per answer (selected answers)

All responses which are selected are evaluated. Partial credit can be awarded for correct answers selected as well as deducted for incorrect answers. However, the lowest possible total score is 0.

#	Question/ task	Answers	Points if selected
1	Select the correct response(s)!	Correct option 1 Incorrect option 1 Correct option 2	1 0 1
2	Select the correct response(s)!	Correct option 1 Incorrect option 1 Correct option 2	1 -1 1

c) Points per answer (selected and non-selected answers)

Each answer is evaluated as a partial solution. Points are awarded for each selected correct answer as well as each non-selected incorrect answer. Points are deducted for each non-selected correct answer as well as each selected incorrect answer. Negative points are not assigned for tasks overall.

#	Question/ task	Answers	Points if selected	Points if not selected
1	Select the correct response(s)!	Correct option 1 Incorrect option 1 Correct option 2	1 0 1	0 1 0
2	Select the correct response(s)!	Correct option 1 Incorrect option 1 Correct option 2	1 -1 1	-1 1 -1

Implementation of the Model Regulations on Conducting and Evaluating Multiple Choice Exams

TU Dresden’s Model Regulations on Conducting and Evaluating Multiple Choice Exams lays out the following provisions for evaluating multiple-choice tasks:

Multiple-choice questions

Multiple-choice refers to questions with multiple options, but only one correct answer. For this purpose, in particular the Choice interaction (Single Choice) is used in ONYX. The Model Ordinance on Conducting and Evaluating Multiple Choice Exams allows for option a) for evaluating these types of questions.

Multiple-response questions

For multiple-response questions (x of n), there are n responses per question, of which x responses are correct, whereby x must be greater than 0 but must be less than or equal to n-1. The question should not give hints as to whether only one or multiple responses are correct.

The maximum final score per task is calculated from the number of points and a weighting factor which reflects the degree of difficulty of the examination task. The maximum number of points which can be earned for an examination task corresponds with the number of potential responses.

Example with 3 response options:

Point = 1
Degree of difficulty/ weighting factor = 0.5
Maximum final score =
point x number of response options x weighting factor = 1 x 3 x 0.5 = 1.5

For partially correct solutions, the points are determined as follows and multiplied by the weighting factor:

One point is awarded

for each correct and selected answer as well as for each incorrect and non-selected answer, so whenever an actual response corresponds with an intended response.

No point is awarded

if no actual responses correspond with the intended response(s).
if no potential response was selected, even if incorrect answers were also not selected, or if all potential responses were selected, even if some of these were correct.

#	Question/ task	Answers	Points if selected	Points if not selected
1	Select the correct response(s)!	Correct option 1 Incorrect option 1 Incorrect option 2	0.5 0 0	0 0.5 0.5
2	Select the correct response(s)!	Correct option 1 Incorrect option 1 Correct option 2	0.5 0 0.5	0 0.5 0
3	Select the correct response(s)!	Correct option 1 Correct option 2 Correct option 3	not permissible – not all response options may be correct
4	Select the correct response(s)!	Incorrect option 1 Incorrect option 2 Incorrect option 3	not permissible – at least one response option must be correct

In ONYX, this can be realized with a Choice interaction (Multiple Choice), but the Hotspot, Hottext and Matrix interaction also offer comparable options. The Model Regulations on Conducting and Evaluating Multiple Choice Exams allows for option c) for evaluating these types of questions. Specifically, the following settings in the “evaluation” tab in ONYX must be adjusted for each task: