Statistical data sources for education policy analysis: the inner workings of the famous PISA

-

Zoé Lerouge

These past years, the Project on International Student Assessment (better known as PISA) launched in 2000 has taken more and more importance in the Education Ministers’ agendas throughout the OECD countries. By ranking the countries according to their students’ performances, the international study creates pressure for reforms among numerous policy-makers.

To understand more precisely how does PISA works and what it really says about our education systems, we had the pleasure to receive Francesca Borgonovi, policy analyst at the OECD, in our SciencesPo Master of International Affairs Comparative Education Policy class. She revealed to us the inner workings of the famous test.

Ms Borgonovi, to set the scene for our readers, could you first tell us more about large-scale educational assessments (LSEAs)?

Sure! Large-scale educational assessments, studies containing direct measurements of individuals’ cognitive proficiency, have become an increasingly useful tool to evaluate the quality of education systems worldwide. The main challenge for international LSEAs is to provide information that is comparable across countries, that is relevant to identify factors that can help explain differences across population subgroups and education systems, and to inform on changes over time.

This is why the OECD works with countries to develop LSEAs such as PISA and PIAAC. They are unique because they permit to compare the results between different countries, in a same country between different time periods, between countries across time and also between countries across time taking into account multiple variables.

Could you tell us what differentiates these different LSEAs?

If we compare the OECD PISA study with studies such as TIMMS and PIRLS that also assess the reading, science and mathematics skills of student populations we can identify a number of parameters that distinguish them. For example:

- Some are grade-based whereas others are age-based, which makes a difference notably in countries where there is grade repetition.

- Some are curricular based (individuals are asked to reproduce what they are taught in class) whereas some are problem solving based (individuals are asked to extrapolate from what they have learnt and to apply their knowledge in real life situations).

What they have in common is that they are school-based and therefore cannot be used to capture the outcomes of children who dropped out of the education system.

Now, I propose that we move on to the study of PISA in particular. Can you remind us what are the main characteristics of the assessment?

PISA is a triennial school-based, age-based (15 year-olds), problem solving based assessment that is done every 3 years in over 60 countries with the latest study implemented in 2015.

The assessment is organized in two parts. First, students sit a two-hours test designed to measure their performance in sciences, mathematics and reading. Then, they have 30 minutes to fill a questionnaire about their background.

What have been the main changes in the PISA test lately?

Well, there have been several changes:

- From 2000 to 2015 the assessment has progressively moved from pen and paper to computer based. It means that new skills can be measured: for example, reading paper-based texts is different from reading digital texts.

- In 2012, a new assessment of non-curricular problem-solving skills was administered together with an assessment of financial literacy.

- In 2015, a module on student well-being was introduced in the background questionnaire.

Why does PSIA occur only every 3 years?

There are different reasons for this choice. First, education systems don’t change so rapidly so it wouldn’t make much sense to assess their performance every year.

Secondly, and this is an important reason, it takes a lot of time, money and human resources to set up a PISA test and these constraints does not permit to do it every year.

Comments

Post a Comment

Popular posts from this blog

A Proposed Education Reform Program for France

-
Image
Abraham Collier (contributors: Elisa Soyer, Lukas Meeth, Trish Suresh) The education system in France is one of the oldest and most admired in the world, with strong performance on PISA and other international standardized tests. However, those same PISA scores indicate a central weakness in France’s primary and secondary systems: inequality based on social background. Smart students from less wealthy families simply don’t have the same opportunities that they do in an average OECD country, and students from rural families are less likely to graduate from upper secondary than their urban counterparts. Potential Drivers of Inequality To understand possible drivers of this inequality, a team of master level students from the Paris School of International Affairs recently undertook an examination of the French system relative to other OECD peers. Their research yielded the following areas of divergence: 1.      Resource allocation between schools an...
Read more

Equity and Quality in education: a spotlight on Korea

-
Image
Chiara Maria Achermann and Jacques Rosenberg   Korea’s focus on — and success in — education Korea’s education policy has proven to be a benchmark for many countries. Taking a closer look at what Korea does in practice can be useful for understanding what works and what doesn’t in education. According to the OECD, “almost one in five 15 year-old-students across OECD countries does not reach a minimum level of skills to function in today’s society, and there are large performance differences between students of different socio-economic backgrounds” (OECD, 2015, 44). We can agree that a child’s success depends on equity in and quality of education. But what does this mean? Basically, equity and quality means fairness and inclusion. Fairness means that no personal or social circumstances should stand in the way of a child’s potential, curiosity, and path to future success. Inclusion implies that every child should attain at least a minimum skills level (OECD, 2015, 43). ...
Read more

Educational Improvement: What Role for the EU?

-
Image
Christa Rawkins International and transnational organisations have a growing influence on national education policies.  The EU needs to strengthen its position to avoid being left behind. Across the world, national education systems face the same key challenge: how to prepare students for an unpredictable, globalised future. The global governance of education therefore plays an increasingly significant role in international discourse. As such, international organisations have more and more influence; indeed, Jakobi (2009) refers to them as “the main drivers of educational change”.  In this increasingly populated and active educational landscape, what is the role of regional organisations? This post considers the European Union’s direct activity and influence upon the national education policies of community members and offers approaches to consolidating this position. While education has always been a national responsibility, some European-wide initi...
Read more