Software Evaluation

Where in the U.S.A. is Carmen Sandiego?

Oregon Trail

Overview

We chose Carmen Sandiego and Oregon Trail because both are relevant to our project in terms of content and approach. Both teach elementary-school level American history. Our project's approach is similar to Carmen Sandiego's: to motivate learning historical facts and map skills through solving a mystery which requires traveling through the U.S. (or, in our case, the colonies). We wanted to review Oregon Trail because that is the one piece of educational software that Dianne is familiar with, and she tends to talk about the program we are writing for her in terms of Oregon Trail. Finally, both of these pieces of software are valuable because of their popularity with students. What makes them popular? Does their popularity have anything to do with their educational value (or lack thereof)?

Where in the U.S.A. is Carmen Sandiego

The book/playground/game triangle

Carmen Sandiego has the potential to be near the middle of Andy Van Dam's triangle. In addition to the obvious game component (tracking down a criminal), the new version of Carmen Sandiego incorporates the book/playground corners of the triangle by allowing users to leave the mystery and go into "Explore" mode. In Explore mode, a cartoon guide explains the history of the current state to the user. Users may travel to any state without penalty while in Explore mode, which brings in the playground aspect -- however, the lack of activities (other than continuing with the guide's tour or moving to a different state) makes it seem more like a book. Users can also find extra information through the online almanac, which contains pictures and information for each state.

Incentives to learn

Although Carmen Sandiego provides various methods for learning more, there is little incentive to explore. The game does attempt to provide an incentive -- you can verify your warrant with a guide, if you have explored earlier in the game. But this is not a very compelling incentive since the guide does not give you any extra information and it is fairly easy to verify information on your own.

The online almanac is one feature that does provide some incentive for users to look up information rather than guessing blindly. The "find" feature, which lets the user look up a specific clue, incurs a penalty, but there is no penalty for browsing through the information for the different states. For example, if I received a clue that the suspect had gone to the Great Dismal Swamp, I could look at my options for where to go next and look up each state. By doing this, I would encounter interesting facts and pictures for each state.

Time is measured in the amount of battery power used up by the crime computer. Battery power is used for traveling and asking questions. If the user solves a case successfully, her total number of cases solved goes up by one. Because there is no point system related to how much battery power is used, it is easy to guess randomly instead of looking up clues in the almanac. The user can make several wrong choices and still solve the mystery in the amount of time allotted. But users who need more clues (and therefore need to use up more power) in each state, may need to look up answers in order to get them right. In other words, there is some incentive to look up answers rather than guessing, but this incentive diminishes as the user learns the answers to more clues.

Carmen Sandiego relies on repeating clues, which users will hopefully remember the next time they play. This strategy is effective because the game lends itself to being played over and over. The game is different each time -- the user chases a different criminal, goes to different places, and sees different cartoon sequences in each state. In addition, the reward for solving a case successfully makes users want to play again. There are several levels of detective titles -- for example, if you solve 4 cases you may become an Investigator, and if you solve 8 you can become a Senior Investigator. (numbers are approximate) When the user solves 40 cases, he gets to go after Carmen Sandiego herself. Because the game provides so much incentive to play again, users end up memorizing some of the information they have used to solve clues in the past.

How it relates to our project

Carmen Sandiego's most educational component is geography. It teaches geography effectively by providing map-based navigation -- the state lights up and its name is displayed when the user clicks on it, and lines are drawn between the current state and the staet being traveled to -- and by providing directional clues (e.g., "He went to the state that's southeast of Iowa.") We can incorporate both of these strategies into our program (and perhaps include a compass).

The purpose of Carmen Sandiego is not to teach history conceptually, but to teach facts about the different states. It achieves its goal, but users are not left with any historical understanding of the U.S. Our program is also supposed to reinforce facts taught in the colonial history unit, but we feel that a purely fact-oriented approach is problematic because it does not lead to understanding. Therefore, we should not completely adopt the Carmen Sandiego approach of simply throwing facts at users. This is why we initially came up with the idea of having users answer questions or solve puzzles (which will be chosen randomly out of a set of puzzles) in order to get clues -- that way, we can incorporate different kinds of material and will make it more difficult to simply "learn the game" by figuring out where to click in order to win. Our strategy will provide students with a variety of material and will hopefully provide incentive to learn, but will it increase understanding? After observing the class and reading class materials, we concluded that teaching history in terms of cause-and-effect relationships is too complex for them. Therefore, we are focusing our efforts on providing a variety of content and making the puzzles entertaining enough that the students will want to work on them, rather than trying to give them an understanding of colonial history.

We were debating about including online help in our program; in Carmen Sandiego, the online help (without penalty) is a good thing. It encourages students to learn the material instead of guessing, as discussed above. This would probably work well in our program too, since we do not expect the students to know all the answers.

As you progress and solve more mysteries in Carmen Sandiego, you remain at the same level -- that is, the clues do not get more difficult, you have the same amount of time, etc. We found this to be a negative aspect of the program. We would like to make our program more challenging as you get better at it.

From each state, Carmen Sandiego gives you a limited number of choices of where to go next. If you make a wrong guess, that state reappears in the list but is marked with an X. This makes it easy to guess, because the list narrows itself down as you make more wrong choices. On one hand, this is a bad thing because it encourages guessing. On the other, if students make a wrong choice simply because they do not know the correct information, it may be frustrating to have the same number of choices each time without any being taken off the list. Because the students we are dealing with have trouble reading and seem to have a low level of understanding of the material, we should probably make the game easier by taking wrong guesses off the list.

Oregon Trail

This is intended to be more of a simulation/playground type software package. In this way it suffers from the flaw these types of software are susceptible to, that there is little motivation to explore. One can complete this game, although it will be very dull to do so, without ever looking in the index, or trading at the trading post or talking to anyone. There is very little the game does to encourage you to do so. Perhaps the idea is that if kids are curious about things they see in the course of a game, they'll look them up and remember them better because of that (a la Schank and Cleary) but it is unclear how the game will reach out to those unmotivated to look up things on their own.

In this way, it is reasonable to expect users to learn that the people on the Oregon Trail had rough and unpredictable lives, traveled in wagons, and had to hunt and trade to survive. It is unclear, however, whether a user would learn any more specific facts or concepts. The historical content is buried in the glossary and in conversations with locals, neither of which are compulsary. The focus seems to be on buying and trading resources.

One of the positive aspects of Oregon Trail is that it does incorporate this idea of real consequences for choices made, which are the same as those one might have faced on the actual trail. If you don't buy enough food then you die. Also, there is incentive for the user to play the game until it is completed, since the game time is relatively short (a full game can be played out in about 15 minutes) and there is a clear goal (to make it to Oregon).

The interface for the game is fairly intuitive. Clicking on things along the border brings up separate windows. It is, however, a little confusing as to which one of them brings you back to the main screen. It is good that you can interact during the game, even if the only things you can do are hunt and check your status.

Other observations:

  • Resting is played out in "real" time. That is, if you want to rest for 3 days, you actually have to wait about 30 seconds before you can move on. This makes the game drag on and may bore children.
  • Most of the game is played out watching a wagon move as scenery scrolls by. This is also fairly boring, although the purpose may be to get kids to explore other aspects of the game (the glossary, talking to people, etc).
  • There is very little interagetion, and the user does not have much control over what happens. A member of the player's party may get sick and die without any warning.
  • The point system is irrelevant since the only time points matter is after the game is over (when they are calculated). There is no way of knowing how well you are doing during gameplay.
  • The only "fun" parts of the game are hunting and navigating down the river at the end -- neither of which has educational content.