For better or worse, the results of national and international assessments often shape policy decisions; effects trickle down to local schools
On the face of it, the National Assessment of Educational Progress (NAEP) data released in late September were not remarkable. They showed small gains in reading at both grade 4 and grade 8 and a small narrowing of the achievement gap between white and black students. (For details about the findings, visit the NAEP website at http://nces.ed.gov/nationsreportcard.)
What was remarkable, however, were the many different interpretations the data drew from various groups. Some used the results to trumpet the value of No Child Left Behind. Others used the same data to decry its lack of success. Still others used the results to call for changes. (See the sidebar, “Same data, different interpretations,” on page 4 for details.)
It’s difficult to determine which of these interpretations, if any, represents “the truth.” This article simply spotlights the fact that policymakers are using the NAEP results as support for setting policies that will affect education at the national, state, district, and school levels.
Many countries throughout the world use the results of national and international assessments to help shape education policy. Two assessments in particular generate great international interest: the Programme for International Student Assessment (PISA), which tests 15-year-olds’ skills and knowledge; and the Progress in International Reading Literacy Study (PIRLS), which targets students in their fourth year of schooling.
With this in mind, we contacted several IRA leaders throughout the world to see how the relationship between such assessments and education policy has played out in their respective countries. We received the following replies, condensed in some cases to fit the space available.
National assessments of levels of attainment in reading have been going on in the United Kingdom since 1948, says Greg Brooks, a longtime United Kingdom Literacy Association leader. In recent years, he noted, the data show a substantial rise in reading achievement at age 11 between 1996 and 2000, then a plateau. At age 15, the data show a gentle rise between 1997 and 2004, then a plateau.
Average levels remained remarkably constant for many years beginning in 1960, Brooks said, which led to the influential Bullock Report in 1975. That report made numerous recommendations for teaching and teacher training.
Still, achievement levels did not rise. That prompted the government to introduce a national curriculum in 1989 across all subjects. In addition, teachers were required to undertake five days a year of inservice training, and initial teacher training was reformed to take account of the national curriculum. National tests also were introduced in the mid-1990s at various age levels.
“Not content with this, the incoming Labour government in 1997 introduced a National Literacy Strategy for primary schools in 1998 and extended it to pupils aged 11 to 14 in secondary schools in 2001,” says Brooks. “Most recently (since 2006) there has been renewed interest in the use of systematic (mainly synthetic) phonics instruction in the initial teaching of literacy. This has been accompanied by government-mandated changes in the initial training of teachers. It is too early to say what the effect of this will be.”
All in all, Brooks believes that the government has made valid use of large-scale assessment data in the United Kingdom and that “most of the resulting policies have been beneficial, except that there is too much testing and this has raised anxiety levels among children.”
In Israel, international assessment has had “a great influence on the policymakers,” says Shlomo Alon, president of the Israel Reading Association. As a result of international assessment data, Quantitative Math became a central issue in the education and professional development of math teachers.
“The media played a very significant role in maintaining pressure on the policymakers to trace accountability concerning the relatively low results of Israeli students in the international assessments,” says Alon. Educators are expected much more to “bring results” in PISA and PIRLS, and inservice training is much more directed now toward achievement. “The resulting policies have been only partly beneficial,” he concludes.
In New Zealand, analysis of national and international data has been used to tailor policies that address issues relating to achievement; effective teaching; the quality of schooling; and student, family, and community engagement, reports Heather Bell, a longtime leader of the New Zealand Reading Association.
New Zealand has a number of well-constructed achievement tests, standardized for that country, but they are not compulsory. “The vast majority of the education profession resists any efforts to make these tests compulsory but supports the use of data for diagnostic purposes,” Bell says.
In addition, data from the National Education Monitoring Project provide systemic information about educational progress and offer teachers practical and relevant suggestions that support learning. “Most teachers constructively and deliberately use achievement data from these various sources for formative and summative purposes,” Bell says.
To complement the use of hard data, the Ministry of Education has commissioned a series of research-based Best Evidence Syntheses. The Education Review Office (ERO), the governmental evaluation agency, also has developed a set of evaluation indicators. These are used when the ERO undertakes the external evaluation of schools (about every three years). “All of this information is readily available to schools,” Bell says, “thus enhancing professional learning at the macro level, making accountability transparent and relevant.”
At the school level, principals and trustees set and monitor achievement targets based both on the National Administration Guidelines and on school needs, “thus having education that suits learners and promotes learning,” Bell says.
The Ministry of Education has invested millions of dollars in focused professional development for teachers, significant in a small country, Bell adds. This professional development includes the following, and is supported with a wide variety of additional activities as well:
centrally funded and managed programs and funding for third-party providers of these programs
schooling improvement initiatives for at-risk schools
School Support Services (advisory services) baseline funding
New Zealand has a small, unified education system with a strong national curriculum spanning all the school years. “Teachers have been involved in the design and construction of this curriculum so they have a greater sense of ownership of the philosophy, content, assessment, and overall requirements,” Bell says.
Most teachers find these various initiatives worthwhile and useful for improving their teaching practice. “One of the most salient features of the New Zealand education system is the school-based responsibility of teachers to make professional decisions about learning for their students, to use the nationally provided, high-quality reference materials to meet the needs of their students, and the expectation that they will be ‘data driven’ in meeting those responsibilities,” she concludes.
Renate Valtin, a professor of education at Humboldt University in Berlin, offers these observations about German schools and their use of PISA and PIRLS data.
Valtin notes that Germany’s mean performance on the reading literacy scale in PISA is significantly below the Organisation for Economic Co-operation and Development (OECD) average. In PIRLS, Germany placed 11th out of 35 countries, but only slightly above the average of other OECD countries that took part in both PISA and PIRLS.
In addition, the gap between the lowest and the highest achieving students in Germany on PISA is larger than in any of the other participating countries. Furthermore, the relationship between social background and performance is the largest in Germany.
In PISA, 13% of German students read at the lowest proficiency level, and almost 10% are not proficient even at this level. “This means that almost one-quarter of the 15-year-olds in Germany can read at only an elementary level, as opposed to the OECD average of 18%,” Valtin says.
After the publication of the PISA results in 2001, the Standing Conference of Education Ministers in the 16 German States defined seven primary areas to address. The ministers recommended the following measures, not all of which have been fully realized:
the improvement of language proficiency in preschools
improved links between preschools and primary schools in an effort to attain an early school entrance
the improvement of primary school education and general improvement of reading proficiency
effective support of educationally disadvantaged children—in particular, children and youths of immigrant families
a consistent further development and safe-guarding of class and school quality based on binding standards as well as a result-oriented evaluation
the improvement of professionalism in teaching, particularly regarding diagnostic and method competence as part of a systematic school development
the extension of full-day in-school and out-of-school offers with a view to providing more educational and remedial classes, particularly for disadvantaged students and gifted students
Overall, it is clear that national and international assessment results play an important role in helping to shape education policy in many countries. Used properly, the data can support policy decisions that effectively serve the needs and interests of schools, teachers, and students.
As Keith Topping of the University of Dundee in Scotland pointed out in an article titled “PISA/PIRLS data on reading achievement: Transfer into international policy and practice,” (The Reading Teacher, volume 59, No. 6, pp. 588–590): “In the constant activity and pressure of everyday educational life, practitioners and policymakers rarely find the time to pause and look beyond their own backyard. However, those who do not look to the horizon from time to time can become permanently shortsighted. The PISA and PIRLS studies offer an excellent opportunity to take a broad perspective—to stand on a higher hill—and to see more clearly what each region or country is doing well and where there might be room for improvement.”
Global to local. (December 2007). Reading Today, 25(3), 1, 4.
Journals
Reading Today