Making Hay from Wheats: A Classsourcing Method to Identify Misconceptions
Siddhartha Prasad, Ben Greenman, Tim Nelson, John Wrenn, Shriram Krishnamurthi
Koli Calling International Conference on Computing Education Research, 2022
Abstract
Novice programmers often begin coding with a poor understanding of the task at hand and end up solving the wrong problem. A promising way to put novices on the right track is to have them write examples first, before coding, and provide them with feedback by evaluating the examples on a suite of chaff implementations that are flawed in subtle ways. This feedback, however, is only as good as the chaffs themselves. Instructors must anticipate misconceptions and avoid expert blind spots to make a useful suite of chaffs.This paper conjectures that novices’ incorrect examples are a rich source of insight and presents a classsourcing method for identifying misconceptions. First off, we identify incorrect examples using known, correct wheat implementations. The method is to manually cluster incorrect examples by semantic similarity, summarize each cluster with a potential misconception, and use the analysis to generate chaffs---thereby deriving a useful by-product (hay) from examples that fail the wheats. Classsourced misconceptions have revealed expert blind spots and drawn attention to chaffs that seldom arose in practice, one of which had an undiscovered bug.
Paper
These papers may differ in formatting from the versions that appear in print. They are made available only to support the rapid dissemination of results; the printed versions, not these, should be considered definitive. The copyrights belong to their respective owners.