JSET ejournal
The Comprehensive Evaluation
of Professional Development Software:
A Critique of Methodology
Carl J. Liaupsin
University of Arizona
Distance education alternatives, such as Web-based courses
and interactive CD-ROMs, are becoming increasingly important
methods for providing professional development training in the
area of special education. The trend toward distance education
alternatives has been driven, in part, by the need to train large
numbers of personnel in specific procedures and techniques. For
instance, the Individuals with Disabilities Education Act (1997)
includes language that suggests the need to provide training
in functional behavioral assessment and behavior intervention
planning to virtually all school-based personnel (Yell &
Katsiyannis, 2000). Traditional professional development models
contain a number of inadequacies that make such large-scale training
efforts difficult or impossible. These include the difficulty
of delivering a consistent message across large groups, the high
cost of obtaining substitute teachers, and the problem of scheduling
sufficient training dates (Sailor et al., 1999; Scott & Nelson,
1999). The use of computer-based instructional methodologies
provides one way to meet the need to train large numbers of pre-service
and in-service personnel. Such methods can help schools and agencies
meet the need for standardized, accountable, and accessible training.
Over the past two decades, many applications of microcomputer
technology have been used to support the role of educational
practitioners. These include the implementation of expert systems
(Fink, 1990), data collection devices (Johnson, 1994), Web-based
consultative support (Meers, 1997) and systems-change case study
materials (Bliss & Mazur, 1996). The use of other microcomputer-based
technologies, such as virtual reality (Muscott & Gifford,
1994) have been discussed as having future value in the field
of education. However, few studies have addressed the use of
computers as a platform for professional development activities.
A review of doctoral dissertations in the area computer-assisted
instruction from 1988 to 1993 found only 43 of 322 cases (13%)
in which the beneficiary of the instruction was intended to be
a teacher (Fang, 1995).
When considering large-scale training efforts using computer-based
instruction for professional development, the effectiveness of
the final instructional package is of critical importance. It
is common practice in the field of special education to withhold
recommendations for adopting a teaching practice prior to determining
whether the practice is deemed to be effective. For example,
Sasso, Conroy, Peck-Stichter, and Fox (2001) have suggested that
based on the limits of the current research base, it is inappropriate
to recommend the use of functional behavioral assessment in general
school-based settings. Common elements of the research evaluating
teaching practices include (a) the extent to which the content
and procedures of the instruction are valid, (b) the degree to
which the instructional practice produces reliable results, and
(c) the practicality of the procedure for the given setting and
students. Typically, a piece of educational software can be considered
representative of an instructional practice or strategy and should
be held to the same standards as other instructional practices
and strategies.
