|
Track Analysis
Ronald H. Stevens is designing testing materials that
employ pattern recognition to help students profit from success and learn
from their mistakes
The Interactive Multi
Media Exercises (IMMEX) software system, now used for problem solving in
science curriculums from K 12 to medical school, grew out of the frustration
experienced more than a decade ago by the UCLA School of Medicine’s Ronald
H. Stevens while administering multiple choice tests to his students.
Having concluded that the traditional testing methods were inadequate in
assessing students’ medical diagnostic skills, Stevens began to experiment
with other formats. “In doing so, I discovered the extent to which the
paper and pencil approach limits the depths to which you can go,” recalls
the professor of microbiology and immunology. So he started learning about
computer programming.
 Stevens created the
first version of IMMEX specifically for the UCLA medical school’s microbiology
and immunology students. The program involved patient simulations: Students
were given a case history and, after choosing and obtaining results from
any of dozens of laboratory tests (with each, of course, presenting both
benefits and costs), they were asked to come up with a range of possible
diagnoses. Then, what began as an attempt to move away from multiple choice
testing blossomed into a communitywide effort in education and assessment.
Today, IMMEX is a Windows based software system containing an “authoring
shell” that can be adapted to fit any one of the more than 200 problem
solving scenarios currently available, each presenting a starting condition,
a goal and numerous options on how to achieve that goal.
For example, students of the environmental sciences may confront a situation
in which dead fish are washing up on the shores of a river and be asked
to pinpoint the cause as efficiently as possible. In doing so, the students
can venture down several paths, studying the river’s geography and surveying
the surrounding industries, sampling various sites and, after deciding
where to sample, choosing from among many possible tests. “With each selection
students make, they get another piece of information — data they must interpret
— to help them make their next decision,” Stevens notes. When they get
to a test they don’t understand, they can always log on to the library
for additional information.
When students work on IMMEX, a database is initialized and their movements
are recorded. “It’s important to know how students are going about solving
problems,” says Stevens. An IMMEX program called Analysis delves into the
database of student performances — either individually, or as a partial
or entire group — and allows instructors to view the problem solving processes
students undertook. Lines are connected from one concept domain to another,
showing the path a particular student followed. When looking at an aggregate
group, heavier lines represent more commonly traveled routes. Major concept
areas are represented by colors. So, for example, when attempting to understand
why students aren’t able to solve a problem, instructors can determine
when students failed to obtain the relevant data, or when they obtained
the data but failed to understand its significance. This process also enables
instructors to gauge the relative difficulty and educational value of the
problems themselves.
The next step is to automate the process of evaluating students’ problem
solving strategies. Artificial “neural networks” may be quite useful for
this purpose. The maps produced by Analysis require teacher interpretation,
are not quantitative and don’t provide real time feedback to students.
The neural networks, on the other hand, employ pattern recognition software
programmed to detect the subtle patterns that translate to success, distinguishing
among strategies in a quantifiable way.
“We’re trying to quantify the problem solving process in a way that
is simple and informative enough that teachers will want to use it, but
also research oriented enough that teachers interested in pursuing advanced
degrees will be able to explore the subject further,” Stevens says.
Efforts are also focused on teaching instructors how to program their
own content. Supported by a $2-million National Science Foundation grant,
as well as by grants from GTE and the Drown Foundation, Stevens is training
Los Angeles K 12 teachers in the use of IMMEX. In Stevens’ program, the
teachers attend a monthlong workshop in which they learn how to author
IMMEX programs, with UCLA faculty helping to ensure the quality of the
new programs’ content. “We have K 12 teachers who are now writing grants,
giving talks and returning to school for advanced degrees in this material,”
Stevens reports. In addition, approximately 80 high school and middle school
teachers have attended three day UCLA workshops, where they learn how to
integrate the programs into the classroom setting.
“We’re looking at what students need to know prior to using the program,
what types of follow ups are desirable and what supplementary materials
are needed if students are to record their observations and results,” says
Stevens, who is actively pursuing such questions with researchers in UCLA’s
Graduate School of Education and Information Studies.
In the long term, Stevens would like to explore how to best use IMMEX
to personalize the learning process for students. “Over the 12,000 times
people have run the immunology problems, essentially no two students have
followed exactly the same sequence of tests,” he says. “The learning of
complex skills is very individualistic, and yet we continue to stand up
and lecture to large groups of students as if they were all the same. This
technology can help change that.”
Recharging the Electronic Classroom...
|