Measuring conceptual understanding in mathematics

Researchers: Dr Ian Jones | Dr Camilla Gilmore ...

Project overview


A major challenge for mathematics education research is how to measure pupils’ conceptual understanding with acceptable validity and reliability. Furthermore, opinion is often divided on whether abstract or concrete representations are better at supporting pupils’ conceptual understanding.

This study developed a measure of conceptual understanding using the Comparative Judgement (CJ) approach, and explored how it could be applied in practice.

CJ is a way to assess open-ended and creative mathematical work, which has been found to produce reliable and valid outcomes. It involves no mark schemes and no marking. Instead, two pieces of student work are presented on a screen and the assessor decides which is ‘better’. When many such pairings are shown to many assessors, the decision data can be statistically modelled to generate a score for each student.

Key findings:
  • CJ can be used to evaluate students’ conceptual understanding, and to evaluate the relative effectiveness of different teaching approaches. This finding will enable researchers to evaluate the effectiveness of educational resources and approaches more quickly and validly than has been possible to date. This in turn will provide policy-makers and teachers with better evidence about the relative effectiveness of educational interventions.
  • In terms of the abstract vs. contextualised representations debate, the study found different evidence for different types of mathematics. When comparing two technology-specific approaches for learning algebra, the researchers found that the abstract approach they tested was more effective than a contextualised approach. However, for differential calculus, the abstract approach and contextualised approach that they tested were found to be equally effective.
  • Therefore, the role of abstraction and contextualisation when teaching mathematics is nuanced. Effectiveness depends on the concept being taught, the approach used, and perhaps the age and prior achievement of learners. Importantly, the CJ approach enabled the researchers to overcome the measurement problem that has limited the findings of previous research.

 This project follows on from previous work, including a study by the same researchers, and a recent pilot study demonstrating how CJ could be used to measure learners’ understanding of mathematical concepts.

Team


  • Dr Ian Jones
    Primary researcher
    Loughborough University
  • Dr Camilla Gilmore
    Researcher
    Mathematics Education Centre, Loughborough University
  • Dr Matthew Inglis
    Researcher
    Loughborough University

  • Programme Head, Education
    Nuffield Foundation
  • Director, Education
    Nuffield Foundation

Explore our projects

New

Education | 2020 - 2020

The impact of COVID-19 on mainstream schools in England

View project
In progress

Education | 2020 - 2021

Post-16 educational trajectories and social inequalities in political engagement

View project
Young-boy-wearing-glasses-watches-laptop-school-lesson-with-notepad-The-effects-COVID-19-on-families-time-use-child-development-PROJ
New

Education | 2020 - 2022

The effects of COVID-19 on families’ time-use and child development

View project
Teenage-girl-looks-at-smartphone-next-to-laptop-Growing-up-under-COVID-19-PROJ
New

Education | Welfare | 2020 - 2021

Growing up under COVID-19

View project
New

Justice | 2020 - 2022

Youth custody: Educational influences and labour market consequences

View project
In progress

Education | 2020 - 2022

Competitive effects of free schools on student outcomes in neighbouring schools

View project
In progress

Education | 2020 - 2022

Comparisons of cognitive skills and educational attainment across the UK

View project
In progress

Education | Welfare | 2020 - 2020

Measuring the disadvantage attainment gap in 16-19 education

View project
Early years worker sitting at table with children
New

Education | 2020 - 2021

The ‘common elements’ approach: improving outcomes in early childhood education

View project
In progress

Education | 2020 - 2022

Teacher supply, shortages and working conditions in England and Wales

View project
Teenage-pupil-wearing-woolly-hat-writes-on-whiteboard-The-influence-of-headteachers-on-their-schools-PROJ
In progress

Education | 2020 - 2022

The influence of headteachers on their schools

View project
Two young girls being taught fractions and decimals, learning from a school workbook together
In progress

Education | 2020 - 2021

Teaching fractions and decimals to children aged 3 to 11

View project
New

Justice | 2020 - 2022

Youth custody: Educational influences and labour market consequences

View project
New

Education | 2020 - 2020

The impact of COVID-19 on mainstream schools in England

View project
Early years worker sitting at table with children
New

Education | 2020 - 2021

The ‘common elements’ approach: improving outcomes in early childhood education

View project
Teenage-girl-looks-at-smartphone-next-to-laptop-Growing-up-under-COVID-19-PROJ
New

Education | Welfare | 2020 - 2021

Growing up under COVID-19

View project
Young-boy-wearing-glasses-watches-laptop-school-lesson-with-notepad-The-effects-COVID-19-on-families-time-use-child-development-PROJ
New

Education | 2020 - 2022

The effects of COVID-19 on families’ time-use and child development

View project
In progress

Education | 2020 - 2021

Post-16 educational trajectories and social inequalities in political engagement

View project
In progress

Education | 2020 - 2022

Comparisons of cognitive skills and educational attainment across the UK

View project
Two young girls being taught fractions and decimals, learning from a school workbook together
In progress

Education | 2020 - 2021

Teaching fractions and decimals to children aged 3 to 11

View project
Teenage-pupil-wearing-woolly-hat-writes-on-whiteboard-The-influence-of-headteachers-on-their-schools-PROJ
In progress

Education | 2020 - 2022

The influence of headteachers on their schools

View project
Young-child-blurred-in-background-plays-with-abacus-in-foreground-Early-years-employment-pathways-PROJ
In progress

Education | 2019 - 2020

A systematic review of early years degrees and employment pathways

View project
In progress

Education | 2020 - 2022

Teacher supply, shortages and working conditions in England and Wales

View project
man reading picture book to baby
In progress

Education | 2019 - 2022

Do infants learn new words from educational picture books?

View project
New

Education | 2020 - 2020

The impact of COVID-19 on mainstream schools in England

View project
In progress

Education | 2020 - 2021

Post-16 educational trajectories and social inequalities in political engagement

View project
Young-boy-wearing-glasses-watches-laptop-school-lesson-with-notepad-The-effects-COVID-19-on-families-time-use-child-development-PROJ
New

Education | 2020 - 2022

The effects of COVID-19 on families’ time-use and child development

View project
Teenage-girl-looks-at-smartphone-next-to-laptop-Growing-up-under-COVID-19-PROJ
New

Education | Welfare | 2020 - 2021

Growing up under COVID-19

View project
New

Justice | 2020 - 2022

Youth custody: Educational influences and labour market consequences

View project
In progress

Education | 2020 - 2022

Competitive effects of free schools on student outcomes in neighbouring schools

View project
In progress

Education | 2020 - 2022

Comparisons of cognitive skills and educational attainment across the UK

View project
In progress

Education | Welfare | 2020 - 2020

Measuring the disadvantage attainment gap in 16-19 education

View project
Early years worker sitting at table with children
New

Education | 2020 - 2021

The ‘common elements’ approach: improving outcomes in early childhood education

View project
In progress

Education | 2020 - 2022

Teacher supply, shortages and working conditions in England and Wales

View project
Teenage-pupil-wearing-woolly-hat-writes-on-whiteboard-The-influence-of-headteachers-on-their-schools-PROJ
In progress

Education | 2020 - 2022

The influence of headteachers on their schools

View project
Two young girls being taught fractions and decimals, learning from a school workbook together
In progress

Education | 2020 - 2021

Teaching fractions and decimals to children aged 3 to 11

View project
Maths resit students taking exam
Reported

Education | 2019 - 2020

A new mathematics GCSE curriculum for post-16 resit students

View project
Reported

Education | 2018 - 2018

GCSE and A-level results day project

View project
Reported

Education | 2018 - 2020

Teaching reading: an integrated programme for deaf & hearing children

View project
Reported

Education | 2018 - 2019

Developing a sustainable intervention for disadvantaged children

View project
Reported

Education | 2017 - 2018

Growing up digital

View project
Reported

Education | 2017 - 2020

Educational attainment of children in need & children in care

View project
Reported

Education | 2017 - 2020

Mathematics in Further Education Colleges (MiFEC)

View project
Reported

Education | Welfare | 2017 - 2019

Undermatch in higher education: prevalence, drivers and outcomes

View project
Teacher and young schoolgirl in classroom look at building blocks brain training game together
Reported

Education | 2017 - 2019

Trialling an executive function training intervention for preschoolers

View project
Reported

Education | 2017 - 2018

The effect of retention and turnover on the teaching workforce

View project
Reported

Education | 2017 - 2019

Secondary school choice and academic attainment

View project
Reported

Education | 2016 - 2019

A systematic review of the impact of parent-child reading

View project
Search projects

We aim to improve people’s lives by funding research that informs social policy, primarily in Education, Welfare and Justice. We also fund student programmes that give young people skills and confidence in science and research.

We are an open, collaborative and engaged funder that offers more than money. Through connecting the individual projects we fund, we strengthen their collective impact and give voice to an overarching narrative.

Profile

COVID-19 response

Nuffield-funded social scientists are conducting COVID-19 research in real-time, to capture people’s experiences of the social, cultural and economic impacts of the pandemic.