Dr Yasmine El Masri lecture at Language Testing Forum

Monday, November 23rd, 2015

Dr Yasmine El Masri presented her research on language effects in international tests: the case of PISA science at the Opening Plenary of the Language Testing Forum 2015 co-hosted by the Department of Education, University of Oxford and the British Council on 22 November. The main theme of the event was ‘Assessing general language proficiency: definitions and approaches’. Yasmine’s abstract is below.

Abstract

Large-scale international assessments such as the Programme for International Student Assessment (PISA), the Trends in International Maths and Science Study (TIMSS) and the Progress in International Reading Literacy Study (PIRLS) have increasingly shaped educational policies and reforms in the last decade. While these surveys can provide invaluable insight about particular educational systems, the development and design of such tests has triggered controversy about the quality and the validity of the instruments adopted. Many of the debates revolved around the extent to which language versions of the same test can be developed while ensuring a fair comparison of student achievement across countries. Despite the rigorous quality control exerted on the translation and adaptation processes in international assessments, bias has been detected in some items. With language being a culturally-laden, complex variable which promotes and influences thinking, devising equivalent tests in different languages is a complicated endeavour. Translation effects are unavoidable and hence bias, at some level, is inevitable. This threatens the validity and reliability of the tests and raises questions about the extent to which policies based on international assessments rely on solid grounds. In this plenary session, we will present two studies from PISA 2006, illustrating the challenges of language in the cognitive test and the student questionnaire. We will discuss released items from PISA science tests and student questionnaires and highlight language issues associated with them.

Back to news listing