Notes for Week #5: Evaluating Educational Software I
Roger B. Blumberg, CS92/ED89, 227 CIT
http://www.cs.brown.edu/courses/cs092/cs92.rbb5.html
A few words about Teachers and Computers
Tuesday: What is involved in constructing a
framework for evaluating (and creating) educational software?
Thursday: "The Right-Sized Byte" and alternative
schemes for categorizing and evaluating educational software
For next week
Last week, as the revised project descriptions were coming in, a story
circulated over the AP wire about how few teachers in the US were
"properly trained" to use computers in their classrooms. The story
may have been inspired by a recent report from the Milken Foundation
called "Will New Teachers Be Prepared to Teach in a Digital Age?"
(see
http://www.milkenexchange.org/research/iste_article.html).
Ironically, the point of the Milken Report (that higher education
and policy folks need to recognize their responsibilities and
failings in this area), was often turned into calls for more
requirements for teacher certification and cries of astonishment
about how few teachers had integrated computers into their
teaching practices.
In Teachers and Machines (1986), Larry Cuban noted a
cycle that was common to the introduction (and failure) of classroom
technology in the U.S. since the radio. The cycle, which I've
mentioned before, was:
exhilaration --> scientific credibility --> disappointment --> teacher-bashing
The recent articles about teachers unable to use computers seems
evidence that we have entered the final phase of Cuban's cycle for
computers. But how would teachers have come to know how to teach
effectively with the new technology?
- Colleges of Education have failed miserably in providing new
teachers with even a clue about computers in classrooms.
- Teachers typically have neither the time nor the hardware/software
to research, create and/or experiment with computers as teaching
tools.
- Unlike other classroom technologies, the emergence of computers in
education has brought with it a class of non-teaching "educators"
who skim much of the professional development money that might have
been given to schools and teachers.
- Teachers have received very mixed messages from administrators
and parents about whether they should pursue innovative, experimental
or traditional teaching practices (e.g. curriculum standards vs.
experimenting with technology in teaching), yet the expectations
placed on the creative use of computers by teachers have been
remarkably great compared to other professions
(e.g. doctors, lawyers, police).
- Nearly 40 years ago Paul Goodman wrote, in the preface to his classic
book about youth culture, Growing Up Absurd:
"The sole function of administration is to smooth the way, but in
this country we have the topsy-turvy situation that a teacher must
devote himself to satisfying the administrator and financier rather
than doing his job."
This "topsy-turvy" situation is nowhere better exemplified than in
the use of computers at schools, and this year the difficulty in
getting more secondary school proposals for CS92/ED89 was due in
part to teacher suspicions of "opportunities" concerning technology.
Any good news, Rog? you ask. Yes! It turns out that most teachers who
use computers in creative, successful ways have been the beneficiaries
not of "training sessions" but of inspiration provided by colleagues
and by computers themselves. In CS92 we have had the experience of
inspiring teachers with the programs we've made, and these teachers
then go on not only to become more expert in teaching with
technology but to inspire their colleagues to give computers
a try.
So, if your sponsoring teachers seem technically under-prepared to
give you as much direction as you might like, try to think of them
as primarily under-stimulated by good examples of software and
teaching with computers.
As noted in
Ben's questions to the list,
the readings for Tuesday seem
rather different from each other. This week we begin a discussion of
evaluating educational software, and educational technology generally,
and the (ultimate) goals are these:
- to develop of framework or vocabulary for talking about educational
software.
- to figure out what can or cannot be said about "good design"
independent of the particular audience who will use the program.
- to figure out the relevant questions to ask for assessing "good
design" once the audience is known.
- to figure out how issues of educational philosophy and
cognitive psychology can be integrated into both the evaluation
framework and the structure of computer programs.
Preliminary to work on #1 we read the Ward & Sewell, and the question
I'm most interested in is: "If you were writing this article today,
how would your approach to the subject differ (if at all) and what
would you add or remove from the discussion?
For answers to #2 above, we read the "Screens for Learning" article,
and you might look at
Elaine Chen's outline from last year as well.
Let's look at three examples of educational software and, with them
in mind, brainstorm criteria for evaluation and categorization. The
programs are:
- The Oregon Trail (MECC, 1993)
- Discovery American History on CD-ROM (Prentice Hall, 1997)
- Port of Entry (National Digital Library Project, 1997)
Some questions we might consider:
- What sorts of experiences are the authors trying to encourage?
- To what extent are multiple paths/narratives/styles made possible?
- How does the software relate to the "three assignments" mentioned by
Gardner in chapter 7 of The Unschooled Mind (p. 131-2)?
- What effects do the entertainment/amusement goals have on the
presentation of the content?
- What would be a valid assessment of the effect/effectiveness of
this program?
Naomi's questions for today
raised a number of issues about the schemes used to categorize and
evaluate software. What are your reactions to the schemes in Copeland
and at the Educational Software Evaluation site at TERC (http://ra.terc.edu/SoftwareEval/)?
Naomi also raised the issue of whether visualization is
always an aid to student learning. One question we've not addressed is
whether what we say about learning needs to take into consideration
the difference between experiences in which we are introduced to
certain concepts or sets of concepts, and experiences in which we
are meant to develop an understanding of those concepts. Does this
difference have any value in deciding how to structure software?
Finally, here are a few questions about Schwartz' "The Right-Sized
Byte" that we might consider:
- How does your project fit in the 2x2 content/approach matrix? (p. 172)
- Can you give an example of the confusion of new media with new
pedagogical approaches? (p. 173)
- What is the difference between querying data and having data
query you (p. 174)
- Do CAAS' like Maple and Matlab fit his description of the "new"
software environment he has in mind? (p. 175)
- Do you agree with Schwartz' comment about motivation? (p. 177)
- Have you discussed the questions on p. 177 with your project team?
- Does Schwartz' idea of "aspects of a subject that are complex
enough to be interesting and simultaneously simple enough to be understood,"
begin to address some of the questions we had on Tuesday about
communicating academic content through software? Do you think such
an idea can be generalized beyond a particular set of students and
teachers at a particular time?
Next week we will continue to look at examples of educational software,
and continue our discussion of evaluation as we read the most thorough
study of the use of computers in schools yet to be published:
Teaching with Technology, by Sandholtz, Ringstaff and
Dwyer. Read at least chapters 1-3, and 5 for Tuesday, and 6,9, and 11
for Thursday.
