Sentenced to respond in full sentences

In the EAP program I oversee the curriculum and assessments for, we’re lucky enough not to have to have standardized exit testing. This means we’re free to develop assessments in the format of our choosing, as opposed to having to teach IELTS or TOEFL test prep. So, therefore, that process includes identifying the key linguistic and academic skills which we deem necessary for our students’ future academic success, incorporating those into our EAP curriculum, and then developing ways to assess these skills.

One decision we’ve consciously made is not to have multiple choice questions on reading or listening assessments past the B1 level; question are open-ended. There is ample literature on the pros and cons of multiple choice questions (MCQ) to test receptive skills, and while MCQs can have their merits, we’ve decided to use open-ended questions as we always have our eye on where students are headed–the university, where their expression of the knowledge acquired through reading or listening will rarely be in multiple-choice form. (While there are many contexts within a typical Canadian university where multiple choice exams are prevalent, they seem to be limited to certain types of lower-level undergraduate courses.)  We also expect these open-ended answers to be in complete sentences (though responses on reading or listening assessments are not marked for grammar, spelling, or punctuation).

What’s interesting about this policy is the blowback it creates. Some students, especially from certain countries, have never, ever, had their English assessed in any way other than multiple-choice questions (nor have they ever covered content in an English class that wasn’t standardized test preparation in some way; this affects alternative assessment in the productive skills areas as well. It’s funny how in some students’ minds, English and the whole standardized-testing world–the global IELTS-industrial complex–seem to blur together.) After the first midterm reading and listening tests there is invariably a line-up of students at my door wanting to complain about the format (and, more often than not, how difficult they found it). 

More surprisingly, for me at least, is blowback that will occasionally arise from instructors. Sometimes it seems like they themselves are having a hard time shirking the weight of years of teaching in the long shadow of the IELTS-prep world, where answers are wrong or right, with no judgement required. Other times it seems like they understand intellectually why we value open-ended questions over multiple choice, but underestimate the amount of class time necessary to explain, contextualize, and practice this so that all students can adapt to this new-to-them paradigm, and then balk at the student blowback.

This is just one of the ways that the IELTS-industrial complex and student expectations can pose challenges to curriculum design and assessment in EAP. 

Advertisements

8 thoughts on “Sentenced to respond in full sentences

  1. I completely agree that to assess language proficiencies, multiple choice is less relevant than open-ended questions. However, I would disagree that they don’t have a place in EAP assessment of some sort. One thing we’ve considered is the fact, as you say, that students will face multiple choices exams in their lower-year undergraduate study. Do we ignore the fact that these weeder courses are ones in which our students will be tested the next year? If we are preparing students for success in their undergraduate courses, these are a reality in some disciplines. Though strategies for taking multiple choice tests themselves don’t really assess learnt language, it is an academic skill worth addressing. On top of that, the language within multiple choice questions can be very tricky and require close reading comprehension skills.

    Having said this, I’m in no way in favour of multiple choice anything (except buzzfeed/facebook personality quizzes) in an ideal learning situation (well, maybe to ease marking), nor do I suggest any assessment EAP programs do focus solely on the multiple choice question. It’s just not simply an IELTS-based expectation that fuels their use in undergraduate courses. A happy medium, I’d suggest, is addressing the language used in them, while using them here and there even to break up the monotony of open-ended question exams.

    • You’re right about the prevalence of multiple choice exams in weeder classes, so in this respect, the fact that the majority of our students are future grad students definitely has influenced this policy in our institution. I find, though, that in terms of general strategies for MCQs that aren’t focused on language, many of my students could write the book! They’re experts…

      • Ha ha ha, is this a Canadian term? Gigantic lower-year undergraduate courses, usually prerequisites for entry into competitive programs, that are purposefully very difficult to “weed out” lower achieving students. At our institution first-year engineering classes are notorious for this.

  2. I’m completely with you on this one. As students go up the academic levels, it definitely becomes less and less about right and wrong answers. I teach mainly students arriving for postgrad courses and that’s one of the big adjustments they have to make. We get to the end of discussing a question and they ask “So what’s the right answer?” – having failed to grasp that the whole discussion was the ‘answer’!

    When I was working on Oxford EAP Adv (aimed at students at that level), we had a nightmare writing the answer key for the TB – almost every activity had ‘suggested answers’ and in lots of cases, they turned into a whole commentary! It wasn’t nice and neat, but I think it’s unavoidable.

    • It’s true–it’s not nice and net, but unavoidable. We’re actually in the process of putting into place some controls and cross-marking protocols to try to minimize differences between markers. A bit messier, but necessary.

      • I guess it depends how big the program is too. The pre-sessional I teach on in summer has over 500 students and 40+ teachers. The standardization required just for the productive parts of the assessment is a big enough headache, so to make the comprehension parts open too would make the marking process almost impossible.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s