IAEA Oxford Conference 2018

The team of Project Calibrate will be giving two presentations during the 44th annual IAEA Conference which is hosted by the Oxford University Centre for Educational Assessment. The conference runs between 9 -14 September 2018.  The conference programme will be online soon at: www.iaea2018.org.

Presentation abstracts can be found below.


Assessment of practical science: a policy history in England

Authors: Ann Childs, Judith Hillier, Lena Gray, Ruth Johnson & Jo-Anne Baird

School examinations in England taken at age 16 (GCSEs) are high stakes examinations for the candidates and for the teachers and schools.  The balance of ssessment of knowledge and skills has been a subject of debate for some time, with some arguing that 21st Century Science was too skills-based and others arguing that knowledge-driven curricula led to drilling pedagogy in schools.  Whilst it is widely agreed that practical skills should be part of the curriculum, there have been many difficulties regarding how this can be accomplished reliably and rigorously in a high stakes environment.  Universities have complained that students are now going on to higher education without the laboratory skills that they need.  However, attempts to assess practical science skills through coursework, controlled assessment (undertaken with supervision) and through a written examination have all faced difficulties.  This presentation will trace the policy history of these issues, which goes beyond science assessment, but has heavily influenced it.  At stake is the shape of the construct of science, how it is taught in schools and the form of its assessment.  In data terms, this constittutes small data that are highly influential upon the development of generations’ knowledge and skills in science.  Our presentation will outline the tensions that are faced in the current environment in England, such as plagiarism, cheating by pupils, parents completing homework, narrow testing regimes, constrained views of the nature of practical skills and teacher malpractice.  We seek a discussion with IAEA participants on how these issues of high stakes assessment of practical science have been tackled in other countries around the world.  This work was produced as part of Project Calibrate and it is supported by the Wellcome Trust (grant number 209659/Z/17/Z).


Mapping practical science in guideline documents and its conceptualisation in GCSE assessments.

Authors: Alison Cullinane, Yasmine El Masri, Sibel Erduran, Steven Wooding

This paper reports on an aspect of a larger project currently ongoing in the University of Oxford entitled Project Calibrate supported by the Wellcome Trust (grant number 209659/Z/17/Z)”.  This research project is examining practical science at GCSE level. The term “practical work” is commonly used as an overarching term that refers to any type of science teaching and learning activity, in which pupils are involved in manipulating and/or observing objects and materials in order to understand how science works (Osborne 2015).  However, the assessment of “practical science” in the United Kingdom, has centred around curriculum initiatives (e.g. DfE, 2015) and research investigations (e.g. Abrahams, Reiss & Sharpe, 2013). In order to investigate the methodological approaches promoted to GCSE students, OfQual documents that guide and regulate English examination boards were analysed. Brandon’s Matrix (1994) was used as the analytical lens which presents a wider view of how science is conducted through four methodological approaches.  Utilising this framework, analysis of Ofqual documents and the GCSE required practical activities was undertaken to see which category of methods are most often encountered during the study of science at Key Stage 4 level. The analysis shows that most of the activities fall into two particular categories in Brandon’s matrix; manipulative hypothesis testing and non-manipulative observational approaches.  A further review was undertaken to investigate how the assessment of these required practical activities manifest in science assessments, to obtain frequencies of how the methodological approaches were represented in the assessments. This analysis focused on a representative set of assessments from different exam boards. The relative frequency of different methodological approaches will be presented and the qualitative details of the coverage will be discussed.