4    Diderot Exams and Quizzes

Preamble

You can use Diderot to administer tests, quizzes, and exams. This chapter presents an overview of how these work.

1  Submittable Chapters

The basic idea is to create a submittable chapter on Diderot and typeset the chapter using LaTeX. The supported problem types include

  • multiple choice (including true/false) problems, where the test taker picks one of the choices

  • multiple-answer problems, where the test-taker can choose more than one of the choices

  • free response problems, where the test taker writes their answer in a designated area and if so desired attach an image, and

  • fill-in-the-blanks problems, where the test taker fills in the missing parts of a solution.

Diderot supports using mathematics (via MathJax) in answering questions. Algorithms or code in fixed-font can be included in all problems, including in fill-in-the-blanks question.

2  Time Management

Diderot manages exam time in a way that mirrors in-class exams.

Time Limit and Due Date

Each submittable chapter may be assigned a time limit that specifies the duration of the test. For example, a quiz may be 20 minutes long.

Each submittable chapter may also be assigned a due date specifying a date and a time until which it must be completed.

Taking the Exam

Once a submittable chapter is released, a student can create a submission and take the test. During the test, the students will see the remaining time on their screen. The remaining time is calculated for each student as minimum of the unused portion of the time limit, and time left until the due date of the chapter. This means that if a students starts an exam 30 minutes later that it is scheduled time, they will have 30 minutes less time to complete the exam.

After the due date of a submittable chapter, the students may not take the exam and will not see its contents.

Diderot does not use the release date of a chapter to enforce that the students start only at that time. The students can start as soon as the chapter is released.

Accommodations

Personal accommodations of the students are automatically taken into account when a student is taking the exam. For example, if a student has 50% extra-time accommodation, then Diderot will take this is into account when administering the exam to that student.

The instructor can change accommodations for each student on the Users page.

Make Up Exams

Diderot allows scheduling exams for individual students. To do this, go to “Submissions” and create a submission for the student with the desired due date. The student can then take the exam until the specified due date.

Pausing and Restarting an Exam

If a student is having difficulties during the exam, e.g., due to connectivity issues, the instructor can pause the students exam and restart it. Before they restart the exam, the instructor can adjust the “due date” for the student and add extra time to make sure that they have enough time to complete the exam.

3  Saving and Submitting an Exam

Diderot saves a test-taker’s exam frequently (every few seconds) to make sure that no work would be lost due to problem such as internet connectivity.

At any time, the student can submit their exam by clicking the “Submit” button.

When the time expires, the most recent version of the exam is submitted automatically.

4  Handling Questions During the Exam

Students may ask (and receive answers to) their questions about the exam on Diderot. We have found that this works well, because it forces the students and the course staff to write down their thoughts (this helps student formulate their thoughts and prevents some misunderstandings). We have also observed that the answer to many questions are already stated on the exam and can therefore be quickly linked to by using Diderot’s internal system. This further simplifies the task of answering questions and is likely instructive for the students.

The students can ask questions during a quiz by using Diderot’s integrated question-and-answer system. The best practice is to require the students to ask questions only through the quiz chapter by selecting the atom that they have a question about and creating their question. When the question is created the course staff will be notified and may answer the question. When their question is answered, the student will receive a notification on Diderot and can read the response. To read the To improve consistency between answer and to prevent concurrent answers to the same question, we have found it useful for the course staff to join a shared conference call, e.g., via zoom and coordinate their answers.

On-Site Notifications

For a student to receive notification of an answer to their question, their on-site notifications should be turned on. To turn the notifications on, you may instruct the students to follow these steps.

  1. Go to your user profile (via pull down menu by your profile picture at the top right corner.

  2. Select notifications settings

  3. Check "On-site Notification" for "New question" and "New announcement"

5  Type Setting Quizzes Using LaTeX
5.1  Basic Concepts

The basic construct for writing quiz problems is called a prompt, which intuitively speaking, asks the user to do something.

Prompts

Prompts come in two flavors: questions prompts and answer prompts.

Question prompts include

  • \ask for free-response questions,

  • \onechoice for multiple choice questions with a single correct answer,

  • \anychoice for multiple answer questions with any number correct answers, and

  • \asktf for true-false questions.

Solution prompts include

  • \sol for free-response solution,

  • \solfin for fill-in-the-blanks solution,

  • \choice and \choice* for an incorrect and correct choice, \solt and \solf for true and false answers.

A question typically consisting of a question prompt followed by any number of solution prompts. Any number of questions may be attached to an atom.

Point Values

There are several ways to specify point values to questions, some more straightforward, some more complicated and finer grain.

The author can assign integer point values to each atom. The point value must terminate with a period and should be the first argument to the atom.

By default an atom’s points are distributed evenly across all the prompts, which can result in non-integral, even “irrational” point values, such as \(3.33333333...\ldots 3\).

Example

The heading \begin{problem}[10.] specifies a problem with ten points and no title.

The heading \begin{problem}[10.][Problem Title] specifies a problem with ten points and and the title Problem Title.

Question Factors

The author can specify the real-valued factor of each question prompt. If a question does not have an explicitly specified factor, it is assigned a default factor of \(1.0\).

For example \ask[2.0] specifies a factor of \(2.0\) for this question prompt.

If there are multiple questions in an atom, their factor values are added and the point value of the atom is distributed over the questions in proportion with their factors.

For example, if a problem atom is worth \(9\) points and its two questions prompts have factors \(2.0\) and \(1.0\), then the first question will be assigned \(6.0\) points and the second question will be assigned \(3.0\) points.

Solution Factors

The author can specify the real-valued factor of each solution prompt. When not specified, each solution factor taken on a default value. For free-response solutions, this default factor is \(1.0\). For correct-choice prompts, the default factor is \(1.0\); for incorrect choice prompts the default factor is \(0.0\).

For example \sol[0.5] specifies a factor of \(0.5\) for this solution prompt.

If there are multiple solution prompts belonging to a question prompt, then their factors are totaled and the factor of question is distributed over the solutions in proportion with their factors.

For example, if a question has a factor of \(9.0\) and its two solutions prompts have factors \(0.5\) and \(1.0\), then the first solution prompt will be assigned \(3.0\) factors and the second question prompt will be assigned \(6.0\) factors. The point value for each solution prompt will in turn be determined by the point value of the atom.

5.2  Examples
Example (Free response with a single solution)

The code below will generate a question and a text box for the student to write their answer.

\begin{problem}  
Some flavor text...  
  
\ask Now comes the questions.  Please write your answer into the  
provided space below.  
  
\sol This is the solution. Diderot will leave space proportional to  
the length of the solution for the student to answer.  If you don't  
have the solution, simply use \~ to indicate the length.  For example,  
like this:~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  
\end{problem}  
Example (Free response with a multiple solutions)

A question can have multiple solution prompts. The example below has two free-response solution prompts. For this question, Diderot will display two text boxes for the students to enter their answers. Because this problem does not specify a point value, the factor \(4.0\) of the question prompt becomes its point value. Because there are two solution boxes, each is worth \(2.0\) points.

\begin{problem}  
Some flavor text...  
  
\ask[4.] Now comes the question.  This question requires the student to  
answer two questions.  Please write your answers to each part in the  
provided space below.  
  
\sol This is the solution for the first part. Diderot will leave space  
proportional to the length of the solution for the student to answer.  
If you don't have the solution, simply use ~ to indicate the length.  
For example, like this:~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  
  
\sol This is the solution for the second part. Diderot will leave  
space proportional to the length of the solution for the student to  
answer.  If you don't have the solution, simply use ~ to indicate the  
length.  For example, like this:~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  
  
\end{problem}  
Example (Fill-in-the-Blanks questions)

Sometimes it is easier to formulate a question as a fill-in-the-blanks question. To this, the author can use \solfin (solution-fill-in) prompt, and use the command \fin (short for fill-in) to indicate the parts that must be blanked out for the students to fill in. The following example requires the student to fill in three blanks for a total of \(6\) points.

\begin{problem}[6.]  
Consider the following statement: all prime numbers are odd.  
  
\ask Please indicate if the statement above is correct or incorrect  
and justify your answer.  
  
\solfin To be prime a number, the number must have \fin{no divisors  
  other than itself and one}.  
  
The above statement is \fin{incorrect}, because \fin{if the  
  number is even, it can divide itself and this does not count against  
  it}.  
  
If you have answered incorrect, you can also provide a counterexample  
here: \fin{2~~~~~~}.  (We are adding some additional space to the  
solution so that we don't give any tips about the solution via the  
length of the space).  
\end{problem}  
Example (Multi-Part Fill-in-the-Blanks Questions)

You can ask multiple-part fill in the blanks questions as follows. The following example has 4 points. Because the problem does not specify point values, the total factor of all its question prompts becomes the point value, which is \(4\) points, because each question prompt has a default factor of \(1.0\).

\begin{problem}  
For each of the following recurrences provide a closed form solution.  
  
\ask   
$W(n) = W(n/2) + n$  
  
\solfin $O($ \fin{n\lg{n}} $)$  
  
\ask   
$W(n) = W(n/2) + n$  
  
\solfin $o($ \fin{n^2} $)$  
  
  
\ask   
$W(n) = W(n/2) + n$  
  
\solfin $\Omega($ \fin{n\lg{n}} $)$  
  
\ask   
$W(n) = W(n/2) + n$  
  
\solfin $\omega($ \fin{n} $)$  
  
\end{problem}  
Example (Multiple Choice and Free Response Questions)

Any number of question-solution prompts may be attached to an atom. The example below starts with a free-response question and continues with a multiple-choice question. In a multiple choice question, use * to indicate correct choices. Because the first question has double the factor of the second, the first solution box is worth \(6.0\) points and the second question is worth \(3.0\) points.

\begin{problem}[9.]  
  
\ask[2.0] Give a closed-form solution in terms of $\Theta$ for the recurrence:  
\[  
W(n)=2 W(n/2)+ n.  
\]  
  
\sol  
$W(n)=\Theta(n\lg{n})$  
  
\onechoice[1.0]  
Indicate whether the recurrence is root dominated,  leaf dominated, or balanced.  
  
\choice Root dominated  
\choice Leaf dominated  
\choice* Balanced  
  
\end{problem}  
Example (True-False Questions)

True-false questions are a special case of multiple choice questions and can be written as follows.

\begin{problem}  
  
\asktf  
There are less that 10000 tigers left in the wild.  
  
\solt  
  
\end{problem}  
  
  
\begin{problem}  
  
\asktf  
Rock climbing is an olympic sort since 1990.  
  
\solf  
  
\end{problem}  
\begin{problem}  
  
\anychoice  
Select all the correct statements belowe.  
  
\choice* $2 + 2 = 4$  
\choice $6 \pmod 4 = 4$  
\choice* $8 \pmod 3 = 2$  
\choice* $8 \pdiv 5 = 3$  
\choice $12 \pmod 4 = 2$  
\end{problem}  
Example (Multi-Answer Questions)

Some questions require selecting multiple answers. As in muliple-choice question, use * to indicate correct choices.

\begin{problem}  
  
\anychoice  
Select all the correct statements belowe.  
  
\choice* $2 + 2 = 4$  
\choice $6 \pmod 4 = 4$  
\choice* $8 \pmod 3 = 2$  
\choice* $8 \pdiv 5 = 3$  
\choice $12 \pmod 4 = 2$  
\end{problem}  
Example

The following example illustrates factor distributions between questions and solutions. Because the problem does not specify point values it receives the total factor of \(17\) as the point value.

\begin{problem}[Tricky Factors]  
Give a closed-form solution in terms of $\Theta$ for the following  
recurrences.  Also, state whether the recurrence is dominated at the  
root, the leaves, or equally at all levels of the recurrence tree.  
  
% This question prompt distributes 9 factors over 1.5 factors  
\ask[9.]  
Give closed form for    
$f(n) = 5f(n/5) + \Theta(n)$  
  
\sol[0.5] % This prompt gets 3.0 factors (and points).  
$\Theta (n \lg n)$, balanced.  
  
\sol[1.0] % This prompt gets 6.0 factors  (and points).  
$\Theta (n \lg n)$, balanced.  
  
% This question prompt distributes 8.0 factors over 2.0 factors   
\onechoice[8.]   
  
% This prompt has the default factor of 1.0.  
% Via distribution, it gets 4.0 factors  (and points).   
\choice* Balanced   
% This choice 1.6 factors  (and points).  
\choice[0.4] Leaves Dominated    
% This choice gets 2.4 factors  (and points).  
\choice[0.6] Root dominated     
\end{problem}  
Example

Consider now the same example as above, but this time the problem specifies a point value of \(34.0\)

\begin{problem}[34.][Tricky Factors and Points]  
Give a closed-form solution in terms of $\Theta$ for the following  
recurrences.  Also, state whether the recurrence is dominated at the  
root, the leaves, or equally at all levels of the recurrence tree.  
  
% This question prompt distributes 9 factors over 1.5 factors  
\ask[9.]  
Give closed form for    
$f(n) = 5f(n/5) + \Theta(n)$  
  
\sol[0.5] % This prompt gets 3.0 factors and 6.0 points.  
$\Theta (n \lg n)$, balanced.  
  
\sol[1.0] % This prompt gets 6.0 factors and 12.0 points).  
$\Theta (n \lg n)$, balanced.  
  
% This question prompt distributes 8.0 factors over 2.0 factors   
\onechoice[8.]   
  
% This prompt has the default factor of 1.0.  
% Via distribution, it gets 4.0 factors and 8.0 points.   
\choice* Balanced   
% This choice 1.6 factors and 3.2 points.  
\choice[0.4] Leaves Dominated    
% This choice gets 2.4 factors and 4.8 points.  
\choice[0.6] Root dominated     
\end{problem}