In the recent introduction to this series of blogs, Everything you always wanted to know about SDG 4 data indicators… but didn’t know who to ask, I flagged a dilemma facing researchers and statisticians: how to translate technical processes around indicators into comprehensible measurement?

Never an easy task – but even harder if you have not been part of the discussions in which the indicators were developed from the very beginning. 

Many tools already available about SDG 4 indicators and data

It seems only fair that, as the custodian UN agency for data on SDG 4, the UNESCO Institute for Statistics (UIS) should not only produce the data but also the tools to gather, analyze and report that data. That is why we have developed four easy-to-use statistical publications and data tools for stakeholders on SDG 4:

• the Quick Guide to Education Indicators for SDG 4
• the SDG 4 Data Book
• the SDG 4 Data Explorer  
• the SDG 4 Country Profiles

In this series of blogs, we will walk through step by step the process to capture data for each of the 11 indicators for SDG 4 – the carefully-selected group of leading indicators that will track progress towards each target on education.

Drawing on the Quick Guide, we will look at the definition of each indicator, its concept, how it is calculated, its sources, how it is produced and – very importantly – how it can be interpreted.  

Indicator 4.1.1: how much are children learning?

This blog focuses on SDG 4 Target 4.1: free, equitable and quality primary and secondary education, and Indicator 4.1.1, defined as: the proportion of children and young people (a) in Grade 2 or 3; (b) at the end of primary education; and (c) at the end of lower secondary education achieving at least a minimum proficiency level in (i) reading and (ii) mathematics, by sex. 

The concept for Indicator 4.1.1. is straightforward: measuring the quality of education and learning in two subject areas at three key points: the middle of primary education, the end of primary education and the end of lower secondary education. It is calculated as the percentage of children and young people at the relevant stage of education who have achieved or exceeded a minimum proficiency level (MPL) in reading or mathematics.

On interpretation of the data: the three points of measurement will have their own MPLs that are established by countries in line with globally-defined minimum standards. So for each point of measurement, there will be a threshold, with students either below it, achieving it or exceeding it.

Turning to data sources: the indicator will be based on the results of a wide range of cross-national assessments, such as: the Programme d'analyse des systèmes éducatifs de la CONFEMEN (PASEC), Progress in International Reading Literacy Study (PIRLS), Programme for International Student Assessment (PISA), Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ), Fourth Regional Comparative and Explanatory Study (ERCE 2019) and Trends in International Mathematics and Science Study (TIMSS).

The challenges we face in measuring learning across countries

This is where we hit the first of three measurement challenges. First, these different assessments do not always use the same definitions of proficiency. It is possible to compare the proficiency of Brazilian students to the proficiency of students in Paraguay, because both countries take part in the same regional assessment, but you cannot compare students from either country to students from, say, South Africa or Botswana. Their regional assessment uses different concepts and methods to measure learning.   

There are, as yet, no common global standards for learning assessments. The UIS, however, is working hard on comparability, developing a Global Framework of Reference for Reading and Mathematics. This aims to ensure that all children are being taught what they need to know at different points in their education, to improve the quality of data to inform policies and to enable countries to report their data internationally using common metrics.   

It will take time to develop and build consensus on common metrics. In the meanwhile, the UIS has developed an interim strategy using national data. While these data are not cross-nationally comparable and cannot be officially used for monitoring, they will provide countries and development partners with an overall perspective, or snapshot, of the learning situation at the national, regional and global levels.    

The second main challenge is the need for consistent quality, based on shared technical standards, to ensure that national and regional data are fit for purpose. The UIS is also working on this challenge, as shown by producing Principles of Good Practice in Learning Assessment and related quick guides on learning assessments.  

The third main challenge is the need to accommodate multiple viewpoints: identifying relevant areas of learning that can and should be measured globally; conceptualizing how national and regional data can inform global measurement; and striking a balance between global perspectives on education and local influences and goals.  

Again, the UIS is on the case, developing ways to use existing national and cross-national assessments to facilitate the measurement and reporting of learning outcomes, rather than demanding that one single assessment be used by every country for SDG reporting. We are also exploring innovative approaches, such as ‘social moderation’ to define, for example, what a country sees as a benchmark or MPL. Above all, we understand the political considerations and the need for consensus-building. 

Another challenge is the limitations of the available data, with assessments typically carried out within school systems (referred to as school-based assessments) and therefore covering only the children who are in the classroom. Right now, Indicator 4.1.1. does not cover children who are out of school, so any assessment of their competencies in reading or mathematics has to rely on household surveys.  

The UIS is working to tackle all of these challenges, aiming to help governments measure and monitor student learning outcomes in mathematics and reading against Indicator 4.1.1 and make good use of the data to inform policy decisions, in a way that works with the grain of the national context.

Our over-arching goal is to provide and improve the measurement frameworks that will propel the world closer to a free, equitable and quality primary and secondary education for every single child.

This is the second in a series of blogs on SDG 4 global monitoring indicators. The UIS invites feedback and commentary from all interested readers.