|
Since 1991 we have been using a distance
learning technology to teach MBA students in China.
Our university contracted with two firms in China
to deliver MBA programs to selected executives.
Each student would take 42 semester hours of instruction
(17 separate classes), following exactly the same sequence
of courses as those students enrolled in our regular on-campus
MBA program.
Our objectives for engaging in these distance
learning programs were:
- To enhance the "internationalism" of our faculty.
- To increase the faculty's experience with executive
education.
- To enhance our understanding of a distance
learning technology.
- To improve the overall "quality" of our graduate students.
Prior to these programs few of our faculty had ever been
to Asia and none of the faculty had prior experience with
distance learning. After a review of distance
learning technologies (e.g., correspondence education,
live TV, etc.) we chose TutoredVideo Instruction (TVI),
a technology developed by Stanford University's School
of Engineering in the early 1970s for the purpose of delivering
graduate engineering education to Hewlett-Packard
employees 100 miles distant from Stanford's campus. Stanford
relied upon research of the various distance
learning technologies that suggested that the use of television
as a means of delivering instruction to students, regardless
of grade levels or subject matter, resulted in student
performance that was equal to that of "live" classes.
In the TVI format, live lectures are videotaped and viewed
later at remote locations by groups of students where
discussions, directed by live tutors, are then fed back
to the instructor. A local tutor is trained to direct
the students' learning at the remote location and serves
as the faculty member's local representative, distributing
materials and collecting assignments. The tutor is also
charged with providing answers to student questions. To
do so requires that the tutor stop the videotape when
questions arise, direct a discussion towards providing
answers by other students and, when necessary, forwarding
unanswered questions directly to the faculty. Thus, the
tutor provided the instructor with a direct link to the
remote location experiences. Together the tutor and the
students were expected to "manage" the learning experience
by actively engaging the information presented to them
on the videotapes.
Research on Stanford's success with this program concluded
that:
- the grades of the TVI students were (statistically)
higher than those of the on-campus students whose class
had been taped for subsequent viewing off-campus;
- the grades of the on-campus students were (statistically)
higher than those students who were learning the same
material by watching television with no opportunity
for interacting with other students or the instructor;
- tutors who answered questions directly were less effective
than those tutors who drew students into discussion
when the tape was stopped; and
- in the most effective classes, as measured by their
average grade, the students stopped the videotape to
discuss a point or raise a question approximately every
5 minutes.
A follow up study based on a cross-sectional analysis
of TVI distance engineering education
at eight U.S. universities strongly supported the results
of the pioneering Stanford study. Student performance
in TVI classes was (statistically) higher than that of
students enrolled in the same classes on campus regardless
of whether the instruction was within a degree program
or merely non-degree, "technical" training. Further, the
older the students the higher their performance was in
TVI classes, in direct contrast to on-campus classes where
age and performance are not related. Research comparing
the use of TVI to other distance learning
technologies in delivering engineering courses in Norway
was also in agreement with the Stanford results.
On the issue of cost-effectiveness, TVI has been found
to cost less to deliver than live instruction. Stanford
estimated that their initial experiment cost approximately
9.5% less than the cost of delivering similar coursework
to live students. Research on the use of TVI to deliver
distance learning across a multi-campus
system suggests that it costs from 10-15% less to deliver.
Our faculty felt that the basic TVI model needed to be
augmented with live instruction. We videotaped live classes
for viewing by our Chinese students. Our faculty traveled
to China to hold from 9-11 hours of "live"
discussions with the students either at the beginning
or towards the end of each academic term. For accreditation
purposes our faculty agreed to use a common final examination
to evaluate the TVI students directly against the on-campus
students whose classes served as both videotape subjects
and as control groups. Students were recruited from a
list submitted by their company and were personally interviewed
by a faculty member prior to their acceptance. They were
then given English language and math exams to test their
basic skills. The students entering these programs were
in their mid-30s and each was employed full-time in an
executive position at their company.
The students attended classes (watching and discussing
videos) one night during the week and on Saturday. In
the early days of these programs the videotaped classes
were held in a small, windowless room located in the University's
instructional media complex. Live students shared a small
video monitor where they saw what the instructor was writing
on a pad (and taped via an overhead projector). Microphones
were placed between each set of on-site students so that
both the student's questions and the instructor's responses
were captured on the videotape. Each faculty was required
to submit discussion questions for their tutors and to
require formal weekly feedback from the tutor via fax.
Further, a formal faculty "feedback" mechanism was established
in the form of a videotaped interview with each of the
faculty who taught in the program (thereby permitting
faculty who had not taught to "get on the learning curve"
quickly).
An interim analysis of the student evaluations of the
TVI courses in the first MBA program and a survey of the
faculty teaching in the program identified
significant technical problems (e.g., poor quality video,
inability to hear student questions or comments, etc.)
Chinese students clearly preferred more "live-time" with
the faculty. Therefore, for the second China
program the amount of live time was increased to 12-15
hours, and new overhead cameras, additional microphones,
and more powerful "multidirectional" microphones were
installed. Faculty was encouraged to post their e-mail
addresses on their TVI course syllabi to facilitate greater
interaction between themselves and their TVI students.
Despite these changes, further faculty responses were
discouraging. They felt that the TVI technology and the
enhanced e-mail-based feedback loop was not working, and
desired even more "live" time with their Chinese students.
We also discovered that most of the faculty had no common
means of assessing the performance of the Chinese students
directly against that of their on-campus students and
that the faculty were not using the tutor for feedback
purposes or providing adequate directions for the tutor.
Finally, despite these problem areas the faculty felt
that overall the TVI experience was a rewarding one for
themselves and their Chinese students.
Unfortunately, no formal evaluation of the student perceptions
of the program or their performance was undertaken' during
this interim evaluation. However, Chinese students' course
evaluations continued to note both continued technical
difficulties and renewed requests for more "live" time
in place of TVI.
In 1997 a formal survey of current and past Chinese students
was undertaken, together with an analysis of their performance
in the TVI classes and their student evaluations of those
classes. Approximately 40% of the students that completed
the TVI programs responded to a program performance survey
and an attitudinal survey. We continued to collect student
performance data and the course evaluation data and updated
them on an ongoing basis to see if there were any changes
in either of these measures.
Student Performance: TVI students received slightly lower
grades than did the average on-campus MBA student. Comparing
each of the TVI student classes directly against the on-campus
class that was taped, the TVI students failed to perform
as well. This was contrary to the findings of earlier
research on the TVI technology.
Student Course Evaluations: At the completion of each
course, the TVI students were asked to fill out a specially
developed course evaluation form. The form focused on
three specific areas that dealt with course content, technology
(tapes and taping process), and on-location teaching
by the faculty. We found out that the results vary slightly
among different teaching areas. We attribute
this finding primarily to the type of course and teaching
methodology used in that course (e.g., management courses
were taught using high student in-class participation
vs. accounting courses primarily being lecture based).
The student course evaluations were slightly higher in
primarily lecture-oriented courses.
Attitudinal Responses: In order to determine the perceptions
of the off-site students on our programs and to see if
these perceptions had changed over time (from one program
to the other), we conducted an attitudinal survey of the
graduates of three China cohorts that had
completed their MBA studies.
The questions focused on four major areas:
- whether the students developed the desired managerial
skills;
- whether the tutored-video format was effective;
- whether the institutional support (Chinese companies)
for the program was adequate; and
- whether the program was valuable (impact on career
progress, benefit to the organization, and recommend
the program to colleagues, respectively).
Results in all cases showed high levels of' satisfaction
and general improvement in the degree of satisfaction
over time.
The results of our evaluation of these programs conclude
that our Chinese (off-site) students are very happy with
the TVI technology and proud of the education
that they received. Our faculty also highly valued their
experiences with this technology, as they discovered that
they had to be better prepared to teach a course that
was going to be taped and be shown to students in China.
Further, they highly valued the opportunity to gain firsthand
experience in a foreign country (through on-site visits).
Although our TVI off-site students' performances did not
exceed those of our live students, we feel that this is
due to the fact that we did not devote sufficient attention
to the role that the tutor played in each of our classes.
However, it also important that we learned that the TVI
technology is a low-investment, medium-maintenance, high-leverage
means of delivering distance learning. It
does not necessarily require high-grade video productions.
We also found that one of our existing classrooms could
easily be modified for videotaping, and that an investment
of less than $50,000 resulted in a sufficient number of
cameras and microphones. A trained student assistant was
hired and was utilized for videotape production and editing
and duplication. Since the success of this technology
rests primarily with the tutor, and the faculty use thereof,
program directors at both ends are very vital. At the
production end, the program director must make certain
that the faculty is
- using the technology correctly, using the tutor effectively,
- employing the same evaluation methods as in his/her
live class, and
- supplying the necessary handout materials so the students
at the viewing end are able to participate in a learning
environment structured similar to "live" a class.
At the viewing end, the program director must make certain
that
- the students view the tapes as a group,
- sufficient time is allotted for discussion,
- the integrity of the student evaluation process is
insured, and
- the tutor, in fact, meets his/her responsibilities
for providing feedback to the instructor.
|
