Test Validity (or Justification) and Beneficial/Negative Consequences

Almost finished a submission on a major theme of my Lit Rev: current paradigms in Language Test Validity. I'm looking at what Lyle, (Bachman & Palmer, 2010) has to say about "unintended detrimental consequences". I've got to reframe that for the language test score = indefinite leave to remain discourse. A language test like this is unlike any other. 

For example, say a student needs IELTS 7.5 to get her place at a UK University. Let's also agree, just for now, that IELTS scores have been fully justified/validated, and that the University admissions department knows full well, using past experience, that anyone with less than 7.5 will be unable to keep up, submit essays of the required standard, and will not graduate. Maybe they know that because until a few years ago the bar was set at 7.0, and they had a whole load of mushkila situations with L2 students. 

So our student gets 7.0. This looks like a "detrimental consequence" for her. She really, really wanted that particular course at that particular University. Now she has to take shit from her parents, teachers and friends, and make do with a place at a less prestigious University. But it ISN'T a detrimental consequence for her in the framework we're using. If she'd gone to her 1st choice University, she would have struggled, failed, and wasted at least a year of her life. Whereas, she'll graduate from her second best option, maybe an L1 University. Beneficial consequences for all stakeholders. 

Whereas. With language tests whose consequences are leave to remain or no leave to remain. You can see the beneficial consequences for those who reach the cut-score. But those who fail? Does their lower language ability mean they would have failed as a new Scot? That's the crux of it. I need to dig into the policy and politics of it, to see what arguments were advanced for a B1 cut-score, and how someone who didn't meet that requirement, an A2 learner perhaps, would be seen as sure to fail as a new Scot. 



Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice. Oxford, UK: Oxford University Press.

Kane, M. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1–73

Comments

Post a Comment

Popular posts from this blog

LTPTP XXV: We Shall Overcome

Image
We shall indeed.  It's a lovely wee tune to learn.  I've almost memorized it this morning, the idea being, when I encounter a piano, I want something to sit down and play on the bugger.  I've a feeling that my keyboard's keys are lighter than a proper piano's, and there'll be a bit of effort transferring the skills I've gotten so far.  Probably, at some point this year, I'll be in workplace with a piano.  I'll print off some simple music and scales, and work at it, but it's likely to be in a public place so I want something to avoid making a twat of myself initially. It feels right that this should be the first tune I learn by heart.
Read more

Leech or flatworm? Ants and Swiss Chard

Image
For the 2nd time in a week or so, (1st time regarding dwarf beans ), I'm following Alys Fowler's advice, this time with regard to Swiss chard .  Long story short, after midsummer plants which often bolt if planted in spring will resist the impulse to do so, probably because of shortening days; (I say shortening, but in Glasgow, in July, we're still getting 18 hours of daylight).   I got a packet of Bright Lights , at a good price, (another reason to wait until July before sowing, lots of bargains, I've noticed). But where to sow them? Most of the 2nd bed is under a tarpaulin, but it wasn't quite big enough to cover the whole bed, so the last 5ft or so went under a sheet of plastic, (in the background of the photo in this post, here .) Transparent plastic was a foolish idea, weeds were growing underneath it, so I covered most of it with sheets of metal from the old shed. I decided to leave the tarp, but use the area beneath the plastic for the chard. Who-ah there
Read more

VINEGAR!

That wine I started a while back has turned to vinegar. Well, that answered questions I was asking myself back then, viz., 'Do we get vinegar flies around here?' And, 'Are all these little flies vinegar flies?' The fuckers. I thought I was being a clever recycler, not buying proper air-locks and using the old plastic-and-lacky-band method. Ah well, another step up another learning curve. Anybody need three gallons of white wine vinegar and two gallons of the impossibly exotic strawberry wine vinegar? It's been a 'business day', dealing with officials and filling out forms. Don't get me started! Alex came over tonight. He gave me the Aphonic CD (he's been in touch with them and they sent him several copies). I thoroughly approve of this new trend of bands getting their work known over the internet, by-passing (initially at least) the poisonous attentions of the music industry. Come to think of it, some of the people in that business are rat
Read more