« October 2005 ESL Expert Feedback and Comments | Main | What do You look for in an ESL Teacher? »

A Model for Evaluation in ESL

November 01, 2005

I have written before of the importance of evaluating students’ reactions to the courses schools provide. It is interesting how different constituents of a school view this process.

Hello again.

Teachers worry that adverse comments might be used against them. Students may not understand that the course needs to be assessed against realistic criteria and school directors do not always understand how to use the information gained. I think that the evaluation process, if well handled, has a positive washback on everybody. It helps teachers understand what works and what doesn’t. It can help students understand how the learning process works and it can help directors plan and market courses that relate to the students’ immediate and wider needs.

Kirkpatrick's Four Level Evaluation Model (1994) provides a useful framework for schools to measure the effectiveness of their programs. Level One measures the learner's reaction to the course. It is important to be aware of your students’ perceptions of the course because people learn better when they react positively to the learning environment. To measure their reactions, you need a questionnaire that covers the facilities, materials, schedule, friendliness of staff: all the ingredients that combine to contribute to the learning experience.

Level Two addresses the question: did the students learn anything? To measure this you will need pre- and post-course testing. Pre-testing will identify the gaps in their knowledge and allow specific learning targets to be set. Post-course testing will show how far these targets have been met. It is important to help students be realistic here. Learning takes place at different rates so the setting of achievable goals is crucial.

Level Three involves testing the students’ capabilities to perform the skills learned in real life rather than in the classroom. To test this, you may need to ask for feedback from an employer or a college supervisor. Alternatively, you could have some role-play or simulation exercises where you assess how capable the students are of putting their learning into practice.

Level Four is concerned with the final results. It measures wider impacts such as monetary advantage, efficiency, morale, teamwork. You might therefore try to find out if, by following the course, the students have attained their wider goals: obtaining a place in higher education, getting a promotion at work. Collecting, organizing and analyzing level four information can be more difficult than the other three levels, but the results are worthwhile in helping you to plan further programs.

To summarize: reaction informs you how well the learners responded to your course and overall learning environment; learning tells you how well the course worked in helping students to meet specific targets. The performance level informs you how effectively the learning can be applied to the learner's life as an employee, a student, a tourist etc. And, finally, impact informs you of how the course benefited the students in their wider goals.

Let me know what you think!

Patricia

Trackback Pings

TrackBack URL for this entry:
http://www.esl-school.com/mt-tb.cgi/103

Listed below are links to weblogs that reference A Model for Evaluation in ESL:

» When ESL Students Evaluate their ESL Instructor from ESL Lesson Plan
It's that time of the semester again. Student evaluations of the teacher! You know, have another teacher come into the room for 15 minutes, handout #2 pencils and multiple choice grade sheets while you go do the same for another strange . . . [Read More]

Tracked on November 16, 2005 01:50 AM

Comments

  1. Lee Says:

    Patricia, as a teacher, I appreciated your comments about student's course evaluations. However, I can't help but think about the old "bigger business" is > than "educational standards" equation that you must concede exists in many ESL schools abroad.

    It reminds me of my own ESL experiences overseas, with some ESL schools that is, where the song-and-dance-routine ESL teachers always seemed to fare better with their students' evaluation of them than the brick-n-mortar types who actually tried to teach the course agenda. As a result, the winners of the popularity contest were often asked to return the following semesters while others, who were sometimes far better educators, were asked to leave because the students simply "couldn't have fun" with them.

    Sometimes these teachers were turning out students who couldn't satisfactory on the practice TOEFEL exams, for example, but DID return to school the next semester. By the same token, I actaully saw where sometimes the classes that did do well on their practice CAE/CPE exams didn't return to the school because they heard another school had teachers that took their students to pubs for lessons, etc. Perhaps I'm only speaking from my own experience, but I found the whole evaluation process disheartening rather than a chance to find out where I needed to improve. Most of my colleauges, understandly, absolutely dreaded the entire process.

    I realize that the situation with private ESL schools is that the owners sometimes have to play to multiple sides of an issue: students want the best price for an ESL class in town, the funniest ESL teacher they've ever had and, or course, the best education. Even my own present institution(a state university) brands itself with "quality-affordability-excellence." We are paricularly proud of the boldly vague and trite marketing concept of "excellence!"

    Sometimes, it seems, a few of those things are bound to conflict with the others. With that in mind, how seriously can hiring committees really take student evaluations when, very often, many of them will just tick every box for "very satisfied" on the teacher that told the funniest jokes and "very dissatisfied" for the teacher that gave them homework. Shouldn't peer-review teacher observations perhaps count for more (or at least as much as) than student observations?

    Would be interested to hear your thoughts on my "rant."

  1. PDean Says:

    --------------
    Note from Patricia:

    Hi Lee,

    Everything you say is familiar to me. I agree wholeheartedly that peer observation is a valuable tool. Schools, of course, need to look at the reactions of all the 'stakeholders', so that means students, teachers and sponsors of students. Merely gathering reactions is never enough: those reactions have to be interpreted. The kind of facile student reaction you mention should be recognized for what it is.

    But schools have to be constantly reviewing their course provision, classroom approach etc. simply to remain competitive. I don't advocate any sort of instant reaction to the kind of comment you refer to, but no business today can survive unless it pays attention to quality assurance and you can't do that unless you know what clients appreciate.

    Good to have your input!

  1. Mark Johanson Says:

    Hi, I agree with most of what you discussed in this post. But I was wondering if you could clarify some of the ways to get indirect feedback from those around you without directly questioning the employer or so forth. Thanks so much.

  1. Brenda Townsend Hall Says:

    Mark,

    If you don't wish to approach the employer, then you could devise a questionnaire for the student to send back to you some weeks after finishing the course. You could ask what differences the course has made to the student's work: are there concrete benefits such salary raises or more satisfying work; is the student's self-confidence greater; have compliments been made about the student's improvements; have other people shown an interest in taking a similar course?

Post a Comment

 

 

 
Remember Me?

(you may use HTML tags for style)