PUBLICATIONS Inside Illinois Vol. 21, No. 7, Oct. 4, 2001

New online system available at UI for instructor/course evaluations

By Sharita Forrest, Assistant Editor
(217) 244 -1072; slforres@illinois.edu

Photo by Bill Wiegand Online evaluations The Division of Measurement and Evaluation in the Office of Instructional Resources processes about half a million Instructor and Course Evaluation System forms every year. A new online evaluation system would not only save printing costs, but also would provide added flexibility and data manipulation features not available with the old system. (From left) Eric Keeley and Rob Manaster, programmers in the Division of Measurement and Evaluation, and Chris Migotsky, associate head, have been involved in the develpment of Evaluation ONline.

Teachers who are looking for an alternative to the paper-and-pencil instructor and course evaluation forms may be interested in a new online system under development by the Division of Measurement and Evaluation (DME) in the Office of Instructional Resources.

The Evaluation ONline program, or EON, is similar to the Instructor and Course Evaluation System (ICES) familiar to UI instructors and students. However, EON provides added flexibility and data manipulation features not available with the old system.

The system was pilot-tested with end-of-semester evaluations for two courses during the spring 2001 semester and 16 courses during the summer.

The pilot tests were conducted with online courses such as the master’s degree program in the Graduate School of Library and Information Sciences and the human resources education online program in the College of Education.

EON probably will be available campuswide during spring 2002 for instructors who want to use it, whether their courses are classroom- or Web-based, said John Ory, director of OIR.

One advantage of the online system is that instructors can custom design their evaluation forms by choosing among more than 100 system-provided questions. Pre-designed forms also are available on the system.

Faculty members in the library school who participated in the pilot testing especially liked being able to tailor their questions to suit their own needs, said Linda Smith, associate dean.

The college’s master’s degree students were enthusiastic about the system’s convenience because they were freed from having to complete and mail paper questionnaires, a process that was antithetical to the nature of their courses, which were entirely online, Smith said.

"To me, it’s a very positive step," Smith said about the transition to a Web-based system. "Increasingly, even on campus, students are accomplishing more online. It just enhances ease of use while maintaining the good aspects of the ICES system."

Library school members were enthusiastic about trying an online system because the school requires all its faculty members to conduct end-of-semester evaluations. With 165 students from 41 other states and foreign countries and those students taking various course combinations, the process of collating and mailing paper evaluation forms to the students was labor intensive, Smith said.

The response rate among students in the library school master’s courses was only
50 to 75 percent with the paper forms; the college hopes to increase that rate by using the online evaluation system, Smith said.

Other universities have begun offering online course evaluation systems or have such systems under development, including Brigham Young University, Indiana University and Northwestern University.

Some universities have found that response rates declined with online evaluation systems, Ory said. At least one university mandates completion by withholding students’ grades until they complete the course evaluation.

However, Ory said he doesn’t foresee the Urbana campus resorting to those measures.
In addition to the convenience it offers, an online evaluation system also better ensures respondent anonymity since the responses to all questions, including open-ended ones, are typed rather than handwritten, said Cheryl Davis Bullock, division head of DME.

The EON system is housed on a secure server within OIR, and access is regulated by the university’s Bluestem verification system.

Students are given a two-week window to access and submit their electronic end-of-semester questionnaires. When the semester concludes and grades have been posted, instructors and administrators can then access the results on the Web.

Department heads can view faculty members’ semester-end results individually or collectively. The results from EON are available much more quickly than the ICES summary reports, which take about four weeks to process and mail.

Instructors also can choose mid-semester evaluations under the system, providing feedback that can be crucial to improving a course, Bullock said. Mid-semester evaluation results, for instructors who use that option, are provided to the instructors only and not their administrators.

The new system gives instructors more discretion in analyzing their data than they have had with the ICES forms. With EON, instructors can sort the responses according to different demographic variables, for example gender or class rank, to determine if variations exist between groups.

When EON becomes available campuswide, instructors will continue using the paper-and-pencil ICES forms as they have in the past unless they contact DME requesting to be added to the EON database, according to Chris Migotsky, associate head of DME.

Instructors who use the online system will still be eligible for The Incomplete List of Teachers Ranked As Excellent by their Students. As with ICES results, EON results will appear on the Longitudinal Profiles.

EON has been under development for 3 1/2 years funded by a $110,000 grant from UI Online.

Conference attendees discuss online assessment usage
By Sharita Forrest, Assistant Editor
(217) 334 -1072; slforres@illinois.edu

Despite the air traffic stoppage related to the Sept. 11 airline hijackings, representatives of many Big Ten universities still made it to Urbana for the first Online Faculty and Student Assessment Conference Sept. 13-14.

The event, hosted by the Division of Measurement and Evaluation in the Office of Instructional Resources, enabled staff members of the universities to discuss how their institutions are developing or already using online student placement and proficiency exams and course evaluations.

Staff members in DME are investigating the possibility of online placement and proficiency testing for incoming freshmen, who are required to take placement tests in mathematics, chemistry, English and foreign languages.

The paper-and-pencil tests typically take four to five hours, according to Diana Steele, coordinator of placement and proficiency services in the division.

"That’s a big chunk out of their day," Steele said. "That’s time when they could be touring campus or doing other orientation activities."

According to Steele, research has shown that online testing can cut the test-taking time by 40 to 50 percent. Moreover, online testing saves travel time since students can take the tests at home or at a testing center designated by their university.

Furthermore, online test results are available much sooner than results of traditional paper-and-pencil tests.

Although the UI is investigating the possibility of allowing incoming students to take the tests from home or other sites, security is a concern, said John Ory, director of the Office of Instructional Resources.

Unless students are required to come to a testing center and show identification, there’s always the chance they may have someone else take the test for them, Ory said.

Another consideration is establishing and staffing help lines so that students who run into problems while taking the tests can call the university for assistance.

Steele said she’ll be working with staff members in the math, English, chemistry and foreign language departments to develop placement tests. However, a pilot program probably won’t be available for at least a year.

Some schools, including Indiana University, Purdue University-Indianapolis and Michigan State University, already offer online testing to incoming freshmen. The University of Minnesota conducts more than 90 percent of its mathematics and chemistry placement tests online, according to information presented at the conference by Margie L. Tomsic, program director in Minnesota’s Office of Measurement Services.

Instructors needed for pilot test
The Division of Measurement and Evaluation is seeking instructors who would like to conduct mid-semester and end-of-semester course evaluations for the fall 2001 semester by means of the new Evaluation ONline system.

Instructors interested in participating should contact Chris Migotsky or Cheryl Bullock at 333-3490 or by e-mail at c-migot@illinois.edu or cdbulloc@illinois.edu by Oct. 15.