Giving detailed, contextful rubrics is one of the most important tools you can utilize to increase the accuracy and the alignment of the AI grader.

Each rubric consists of one or more items, with each item assigned a point value and a descriptive label. Rubrics can follow either a subtractive scheme—deducting points for common errors—or an additive scheme—awarding points for correct components. Minimum and maximum total scores can also be specified.

Pensive can generate rubrics for your problem set. If solution pdf is provided, Pensive extracts the solution from the pdf and use the problem and solution altogether to generate the most reasonable rubric items.

If the solution is not provided, Pensive generates the solution and then generates the rubric.

If you prefer rubric items with a specific style, you can also provide a prompt for generating more aligned rubrics.

Constraints is one way to increase the accuracy of the AI grader. For instance, if we were to select "Up to two missing or extra vertices" rubric item from the rubric below, we shouldn't select "More than two missing or extra vertices."

By adding constraints, you can enforce a mutually exclusive rubric items so that we can prevent AI graders making basic mistakes of selecting multiple rubric items from the group.

You can click Add Constraint button and select all rubric items that should be mutually exclusive.

Grading some problems depend on the answer student wrote on the previous question. In this case, you can designate prerequisite problems so that Pensive AI grader can consider answers to those problems when grading the target problem.

You can set Prerequisite Problems in the Detailed rubric view from the rubric page, or by clicking the Edit rubric button from grade review page.

Please refer to Calibration docs.