Generating Programs Trivially: Student Use of Large Language Models
Siddhartha Prasad, Ben Greenman, Tim Nelson, Shriram Krishnamurthi
ACM Conference on Global Computing Education, 2023
Abstract
Educators have been concerned about the capability of large language models to automatically generate programs in response to textual prompts. However, little is known about whether and how students actually use these tools.In the context of an upper-level formal methods course, we gave students access to large language models. They were told they could use the models freely. We built a Visual Studio Code extension to simplify access to these models. We also paid for an account so students could use the models for free without worrying about cost.
In this experience report we analyze the outcomes. We see how students actually do and do not use the models. We codify the different uses they make. Most of all, we notice that students actually do not use them very much at all, and provide insight into the many reasons why not. We believe such experiments can help rebalance some of the public narrative about such tools.
Paper
These papers may differ in formatting from the versions that appear in print. They are made available only to support the rapid dissemination of results; the printed versions, not these, should be considered definitive. The copyrights belong to their respective owners.