News
scoris assessor reliability backed by research
19th November 2009
Organisations using scoris assessor can enjoy unique certainty that marking carried out in the application is reliable. The software has been subject to rigorous research into the impact of ‘mode’ on marker reliability, with an important recent study comparing marking on paper with marking in scoris, for extended subjective texts.
The paper was presented as ‘Marking essays on screen: an investigation into the reliability of marking extended subjective texts’, by Martin Johnson, Rita Nádas, John F. Bell and Sylvia Green at the IAEA conference in Brisbane, Australia in September 2009. It was also published in ‘The British Journal of Educational Technology’ (2009) by the British Educational Communications and Technology Agency and Blackwell Publishing.
This paper is important because a previous paper investigating a similar issue with a different marking application found that such subjective responses could not be reliably marked on screen. The 2009 paper described how twelve experienced English Literature assessors marked two matched samples of ninety essay exam scripts on screen using scoris and on paper. The researchers used an essay question from a General Certificate of Secondary Education (GCSE) examination in English Literature as a focus for study. Essays were 800 – 1500 words long.
RM believes that this research, which contrasts with previous studies, represents the culmination of a long period of system design and careful attention to navigation, flexibility, ease of use, tools to support markers in tracking their thinking. This enables markers to focus purely on marking and not technology, pressures for productivity, finding their way round the responses or struggling to record their thinking as they work.