Look in many school foyers and you will notice the school's academic goals are displayed. They will be framed and say something like this:
Yet how meaningful are NCEA pass rates and such goals?
There are three issues to consider.
First, two schools may have the same pass rate but will not have used the same mix of achievement and somewhat easier unit standards to get the same result.
Second, schools, in particular principals, are canny enough to change their school's mix of unit/achievement standards over time to meet professional performance goals.
Third, schools that don't play the game of changing the standards mix are getting overtaken or devalued by schools that do.
What appears to be missing from recent magazine features is how performance can be influenced by changing the mix of unit standards and achievement standards within subjects. Unit standards, generally speaking, are less challenging than achievement standards. Unit standards permit re-sits of non-achieved tests and finally allow verbal examination whereas achievement standards are stricter. Schools are able to mix unit and achievement standards as they see fit across all subject domains.
Regarding school performance, what is striking is that a principal can appear to move the academic standing of their school, and of particular groups within their school, without an equivalent improvement in teaching standards and objective academic standards.
Schools can be "improved" and the principal and senior management appear competent by changing the mix of standards. For example, if you want to improve numeracy and literacy you ignore the achievement standards and push the unit standards at Year 11. The result is that your school's basic literacy and numeracy rate will leap dramatically in 3-4 years. That, however, should not be begrudged. The NCEA literacy and numeracy standard, 8 credits of each, is a basic level of achievement and can serve as a positive introduction to academic success.
What is concerning though are principals and management teams that change the unit-achievement standard weighting across the whole school. This gives the impression that adroit leadership is lifting the competence of their students. Instead, it is the dilution of subject material and the reduced challenge which is giving an appearance of improving pass rates. Who really benefits? The students or the careers of teachers?
Consider the following real but unnamed schools, A and B. From the 2006 data, School A's NCEA 1 pass rate was 69%. School B's NCEA1 pass rate was 65%. School A was decile 3 whereas School B was decile 6.
All things been equal, which school appears to be better? Indeed, School A is decile 3 and is outstripping School B! School A must be doing something right.
However, when the mix of unit standards (US) and achievement standards (AS) are taken into account the picture changes. Here are the results again with the %mix as (unit%/achieve%)
School A: pass rate 69%, (U55/A44).
School B: pass rate 65%, (U27/A73).
School A achieves their 69% pass rate by using unit standards half the time whereas School B hits their 65% pass rate using unit standards only quarter of the time.
Now how do the two compare?
Further 2006 data for NCEA1 Y11 (formerly 5th form School Cert for 15-16 year olds):
Otago BHS: pass rate 75%, (U32/A68).
Palmerston North BHS: pass rate 80%, (U17/A83)
PNGHS: pass rate 82%, (U12/A88)
Auckland Grammar: pass rate 26%, (U9/A91).
Westlake Boys: pass rate 52%, (U18/A82) decile 10.
Glenfield HS: pass rate 56%, (U30/A70).
Wanganui High School: pass rate 88%, (U42/A58) decile 5.
New Plymouth BHS: pass rate 68%, (U18/A82) decile 7
Wellington College: pass rate 83%, (U12/A88).
Chilton St James: pass rate 99%, (U11/A89)
You will notice two things. Firstly, most schools have pass rates over 60% and that unheralded schools such as Wanganui HS can have pass rates to rival Scholarship factories such as Palmerston North GHS and secondly, the mix of unit to achievement standards varies massively.
It is interesting to follow the changing MIX of unit and achievement standards over time at a single school.
For example, School A above had a NCEA1 pass rate in 2003 of 49% (U45/A55), by 2006 the pass rate was 69% (U55/A45) and in 2007 the pass rate was 70% and the mix (U65/A35).
Notice anything? Between 2003 and 2006 the ratio of unit standards to achievement standards was flipped and in the following year the trend steepened to the point where a mere 35% of credits awarded school-wide at Y11 were"achievement standards".
Naturally, the principal can point to these figures as a success. They will have achieved their performance goals and will trumpet that in newsletters.
Perhaps parents and Boards of Trustees need to be more canny!
Lastly, the consistency of certain schools and their unwillingness to play the game of mixing standards shows through in the statistics. In 2003, Wellington College's pass rate was 79% U5/A95; in 2006, 83% U12/A88; in 2007, 82% U9/A91. In 2003 Palmerston North BHS pass rate was 80%, (U15/A85) and their most recently published 2007 results are about the same with the same standards mix. In comparison, the 2004 New Zealand pass rate was 54%, by 2008 it had risen to 63%.
Is it any wonder that once schools realized they could give the impression of academic improvement that the pass rate has inflated 9-10% over four years?
Have schools such Wellington College stood still or are other schools more cynical and playing the system if allowed by their BOT?
The stats for your school are available here: NZQA School Stats.
Method: the data used above are taken for a particular year, in this case 2006, and "Percentage" display selected. The NCEA1 "pass rate" is the "% of roll achieving greater than or equal to 80 credits (80 credits is the requirement for obtaining NCEA). The unit to achievement standard mix for that year level (Y11) is found in the same column, "% of NQF results gaining credit". Unit standards are awarded as "pass" while "achievement standards" have the passing grades achieve, merit, and excellence. Anecdotally, the unofficial rule of thumb is that "acheive" is in the range of 30-65%, "merit" is 65-80%, and "excellence" is 80-100%.
"We will improve our literacy and numeracy rate" orThis serious document will be signed by the principal as it will be part of the performance goals the Board of Trustees is monitoring.
"We will improve our NCEA1 pass rate"
Yet how meaningful are NCEA pass rates and such goals?
There are three issues to consider.
First, two schools may have the same pass rate but will not have used the same mix of achievement and somewhat easier unit standards to get the same result.
Second, schools, in particular principals, are canny enough to change their school's mix of unit/achievement standards over time to meet professional performance goals.
Third, schools that don't play the game of changing the standards mix are getting overtaken or devalued by schools that do.
What appears to be missing from recent magazine features is how performance can be influenced by changing the mix of unit standards and achievement standards within subjects. Unit standards, generally speaking, are less challenging than achievement standards. Unit standards permit re-sits of non-achieved tests and finally allow verbal examination whereas achievement standards are stricter. Schools are able to mix unit and achievement standards as they see fit across all subject domains.
Regarding school performance, what is striking is that a principal can appear to move the academic standing of their school, and of particular groups within their school, without an equivalent improvement in teaching standards and objective academic standards.
Schools can be "improved" and the principal and senior management appear competent by changing the mix of standards. For example, if you want to improve numeracy and literacy you ignore the achievement standards and push the unit standards at Year 11. The result is that your school's basic literacy and numeracy rate will leap dramatically in 3-4 years. That, however, should not be begrudged. The NCEA literacy and numeracy standard, 8 credits of each, is a basic level of achievement and can serve as a positive introduction to academic success.
What is concerning though are principals and management teams that change the unit-achievement standard weighting across the whole school. This gives the impression that adroit leadership is lifting the competence of their students. Instead, it is the dilution of subject material and the reduced challenge which is giving an appearance of improving pass rates. Who really benefits? The students or the careers of teachers?
Consider the following real but unnamed schools, A and B. From the 2006 data, School A's NCEA 1 pass rate was 69%. School B's NCEA1 pass rate was 65%. School A was decile 3 whereas School B was decile 6.
All things been equal, which school appears to be better? Indeed, School A is decile 3 and is outstripping School B! School A must be doing something right.
However, when the mix of unit standards (US) and achievement standards (AS) are taken into account the picture changes. Here are the results again with the %mix as (unit%/achieve%)
School A: pass rate 69%, (U55/A44).
School B: pass rate 65%, (U27/A73).
School A achieves their 69% pass rate by using unit standards half the time whereas School B hits their 65% pass rate using unit standards only quarter of the time.
Now how do the two compare?
Further 2006 data for NCEA1 Y11 (formerly 5th form School Cert for 15-16 year olds):
Otago BHS: pass rate 75%, (U32/A68).
Palmerston North BHS: pass rate 80%, (U17/A83)
PNGHS: pass rate 82%, (U12/A88)
Auckland Grammar: pass rate 26%, (U9/A91).
Westlake Boys: pass rate 52%, (U18/A82) decile 10.
Glenfield HS: pass rate 56%, (U30/A70).
Wanganui High School: pass rate 88%, (U42/A58) decile 5.
New Plymouth BHS: pass rate 68%, (U18/A82) decile 7
Wellington College: pass rate 83%, (U12/A88).
Chilton St James: pass rate 99%, (U11/A89)
You will notice two things. Firstly, most schools have pass rates over 60% and that unheralded schools such as Wanganui HS can have pass rates to rival Scholarship factories such as Palmerston North GHS and secondly, the mix of unit to achievement standards varies massively.
It is interesting to follow the changing MIX of unit and achievement standards over time at a single school.
For example, School A above had a NCEA1 pass rate in 2003 of 49% (U45/A55), by 2006 the pass rate was 69% (U55/A45) and in 2007 the pass rate was 70% and the mix (U65/A35).
Notice anything? Between 2003 and 2006 the ratio of unit standards to achievement standards was flipped and in the following year the trend steepened to the point where a mere 35% of credits awarded school-wide at Y11 were"achievement standards".
Naturally, the principal can point to these figures as a success. They will have achieved their performance goals and will trumpet that in newsletters.
Perhaps parents and Boards of Trustees need to be more canny!
Lastly, the consistency of certain schools and their unwillingness to play the game of mixing standards shows through in the statistics. In 2003, Wellington College's pass rate was 79% U5/A95; in 2006, 83% U12/A88; in 2007, 82% U9/A91. In 2003 Palmerston North BHS pass rate was 80%, (U15/A85) and their most recently published 2007 results are about the same with the same standards mix. In comparison, the 2004 New Zealand pass rate was 54%, by 2008 it had risen to 63%.
Is it any wonder that once schools realized they could give the impression of academic improvement that the pass rate has inflated 9-10% over four years?
Have schools such Wellington College stood still or are other schools more cynical and playing the system if allowed by their BOT?
The stats for your school are available here: NZQA School Stats.
Method: the data used above are taken for a particular year, in this case 2006, and "Percentage" display selected. The NCEA1 "pass rate" is the "% of roll achieving greater than or equal to 80 credits (80 credits is the requirement for obtaining NCEA). The unit to achievement standard mix for that year level (Y11) is found in the same column, "% of NQF results gaining credit". Unit standards are awarded as "pass" while "achievement standards" have the passing grades achieve, merit, and excellence. Anecdotally, the unofficial rule of thumb is that "acheive" is in the range of 30-65%, "merit" is 65-80%, and "excellence" is 80-100%.
Surely any school using NCEA stats to "prove" it's now a better school would be shooting itself in the foot anyway. To state the obvious, NCEA is not really considered academically challenging anyway.
ReplyDeleteLook in the foyer for the stated academic goals of the school, they'll be there somewhere. That's part of how principals show their BOT that they are hitting professional targets.
ReplyDeleteTargets will be framed in terms of NCEA pass rates.
What other way is there to track improvement, particularly academic improvement for a school?
Achievement standards can be challenging which explains the "low" pass rates of Westlake and Grammar. Instead of chasing a high passrate they have the integrity to offer a challenging cirriculum to their students.
NCEA is more like the paper-mixing within a university degree than what those over 23 years old experienced at school.
It should be appreciated that UNIT standards are a different beast from ACHIEVEMENT standards. UNITS are easier in most subjects and their criteria for assessment is much looser. When you see unit standards dominating the curriculum and a pass rate steaming beyond 70% then I would be very cautious.
For example from above, PNGHS performance of 82% passrate is "better" than Wanganui HS 88% because PNGHS hit 82% with a curriculum that was 88% achievement standards. Chilton St James gets a 99% pass rate of Y11 NCEA1 with 89% of the curriculum composed of achievement standards! Very impressive.