Association for Educational Assessment (AEA) – Europe 2018

Building bridges between assessment and subject matter: An innovative approach to assessing GCSE Practical Science

Authors: Yasmine El Masri; Sibel Erduran, Alison Cullinane, Ruth Johnson.

Conference: Association for Educational Assessment (AEA) – Europe 2018, 7-10 November 2018

Abstract: Assessing practical science skills has been challenging in the United Kingdom and elsewhere largely because of the complex nature of the construct to be assessed and the knowledge and skills that practical science entails. Practical work commonly refers to all forms of activities that provide students with the opportunity of manipulating and/or observing objects and materials in order to build their understanding of how science works (Osborne,2015). The challenge in assessment, especially in high-stakes situations like GCSEs in England, has been to offer students valid contexts in which they can demonstrate their practical skills and knowledge while ensuring the assessments are reliable and fair and while limiting possibilities of cheating and drilling (See Donnelly, 1994). The tensions arising from assessing practical science under high-stakes conditions have often contributed, at least partly, to narrowing the construct down to hypothesis testing and replicating manipulations (Dillon 2008; Reiss, Abraham and Sharpe 2012). However, researchers in science education have argued that practical science is not restricted to experimental work and that it embodies a larger set of skills (Erduran & Dagher 2014). Indeed, scientists like zoologists and palaeontologists primarily rely

on other skills such as the use of non-manipulative descriptive classifications. In addition to procedural skills, scientists make use of higher order thinking such as argumentation that includes evaluating evidence and justifying claims.

Until 2006, practical science was assessed at GCSE level in England through course work.

Concerns about validity and reliability of the assessments as well as the high potential for cheating have led to a move to controlled assessments where students carried out science investigations under test conditions (see Ofqual 2013; SCORE 2009). This practice was costly and time consuming (Ofqual 2013). In addition, it led to drilling students on specific manipulations and teachers ticking boxes to show that students have completed a minimum number of practicals. More recently, the assessment of practical science skills at GCSE level has been incorporated into the pen-and-paper GCSE science assessments. While this approach can limit issues of reliability of assessments, it has raised concerns of validity.

In this presentation, we describe the work carried out by a team of science education and assessment researchers in collaboration with an examination board in English. The work is part of a larger project currently entitled Project Calibrate supported by the Wellcome Trust (Grant number 209659/Z/17/Z). We present a new framework for developing valid practical science assessments based on research carried out in science education. The new proposed framework incorporates Brandon’s (1994, p.63) matrix of scientific methods and allows the development of practical science task that do not run the risk of under-representing the construct. At the

same time, we describe how this approach ensures that valid assessments can be produced where students can demonstrate their skills in carrying out practical work without posing threat to the reliability and quality of the assessment. We also provide task exemplars that align with the Brandon’s matrix to illustrate our framework. The presentation is relevant to the work of researchers in educational assessment, particularly those interested in test development, test validity and high-stakes examinations. It is also relevant to science practitioner and examination boards.