How can the results of PISA and other major international studies be applied to improving the quality of learning? A two-day workshop on this topic will be held on February 19 – 20 in Sarajevo, organized by UNICEF and the Agency for Pre-Primary, Primary and Secondary Education, with the support of the Norwegian Embassy. The Experts from Norway: Ms. Anne-Berit Kavli, Senior Advisor at the Norwegian Directorate of Education and Training and Mr. Julius K. Bjornsson, Researcher at the Department of Teacher Education and School Research will share their expertise with the participants. On the occasion, Mr. Bjornsson answered a few relevant questions.
You reviewed the results of PISA survey for Bosnia and Herzegovina – what can these results tell us about student achievements? What else is indicated by the PISA results?
Certainly, the results for BiH are disappointing, given that they are below the OECD average. However, results in BiH have some specifics, such as low variability of results, which is among the smallest in the survey. The standard deviation in reading is 79 out of an average score of 403. In mathematics, the average is 406 with a standard deviation of 82, and in science the average is 398, and the standard deviation is 77. These cases of small variability are something that should be closely examined. Given the low average, it can be concluded that at least a third of the achievements, at the top, are missing in BiH, meaning that a large group of students is not fulfilling their potential. As for the achievement levels, it is evident that only few students ranked as good readers. Gender differences are similar to those in the rest of Europe where girls outperformed boys in reading literacy. It is similar in math and science.
Can any of the segments be singled out as a good characteristic of Bosnia and Herzegovina?
In BiH, there are also differences between schools, but less than in other Balkan countries. However, these differences are more important in BiH than in other European countries, as they account for a large proportion of deviations from the OECD average. Therefore, it is important to consider and examine the resources and staff in those schools. Consideration should also be given to the basic teacher education as well as their continuing professional development, and the support they have in order to perform their job in the best possible way. Teachers are crucial to improving the student achievements, and without them little can be achieved.
It is important to focus on issues of equality and equity, such as the support available to students who need extra help and to the talented students who need encouragement and motivation. The data show the students in BiH are relatively highly motivated to complete the tasks and results in this segment are higher than in the Nordic countries. This should be the starting point for the improvement process. However, school questionnaires indicate that what the schools lack is not staff but resources and that this has an impact on the teaching process.
Education systems in BiH are very complex and every change requires a lot of time, effort and administration. There are certainly areas that can be improved in a simple way. For example – can teachers apply some of the PISA research recommendations in their work?
It is very important to use the results of PISA 2018 to see what competences BiH students are mastering and what competences they fail to master. It is therefore important to carefully review the tests and documentation that provide detailed information on the complexity of each issue. Detailed analysis of what students do not know will help teachers to identify areas that need to be addressed. The PISA project team should enable access to information wherever possible without compromising confidentiality.
There are also many open-ended PISA items available that can give us insight into how to evaluate them. Not only can we find out which competencies are measured by certain task, but also how they are assessed. This is very useful for further work with students and provides practical information.
This year, we are implementing the PISA 2018 results and, at the same time, we are getting the TIMSS 2019 survey results. Can you clarify how the two surveys are complementary, why it is useful to conduct both surveys along with the PIRLS, as far as the country has such opportunity?
TIMSS measures student achievements in grades 4 and 8 in math and science, while PISA measures the competencies of 15-year-olds who are typically in high school, in three areas. All students in the class, regardless of age, participate in the TIMSS survey, while PISA includes the 15-year-olds who may come from different grades. This allows for TIMSS to isolate the class effects, and for PISA to isolate the age effects.
TIMSS is based on a detailed analysis of the curriculum for 4th and 8th grade and can be considered to measure what students have learned in school. PISA, on the other hand, conceives its measurement on a broad concept of literacy across three areas. PISA is not only about what the students learn in school, but puts the emphasis on how well students can apply their knowledge in real-life situations and in solving problems they may encounter in their later studies and work. In addition, TIMSS allows for measuring the trend in grades 4 through 8.
We can see, based on this, that PISA and TIMSS are complementary studies. The country in which both surveys are conducted has a much broader basis for drawing conclusions about the state of the education system, and can make detailed analyses based on individual differences, differences between schools and between classes at school. The base for reaching conclusions is significantly narrowed if only one research is performed. Also, the two studies to some extent verify each other, which is also important.
PIRLS as a third study is also of importance because it assesses reading competence in 4th grade, and thus expands the picture of achievement in an age where there should be the basics of reading, mathematics and science.
Foto: Đemidžić/UNICEF