Technical Features of the Program
The surveys used as the data collection method for The Balanced Scorecard for Schools program have been extensively quality assured to ensure the information and data they produce is accurate, valid and reliable.
The surveys have been subject to extensive validation and quality assurance featuring the following:
- The comprehensive Survey Framework that guided the development of questions was reviewed and enhanced on the basis of feedback and recommendations provided by the program’s international Expert Advisory Network.
- All survey items were developed strictly in accordance with international standards and guidelines for item design and format
- All draft survey items were subjected to cognitive laboratories
- All draft items were field trialled in two countries and subject to a rigorous psychometric item analysis by our professor of measurement, Dr Margaret Wu.
- The 18 survey forms have been constructed taking into account each survey item’s difficulty to ensure that the forms are each of comparable ‘difficulty’
- The distribution of survey forms to respondents is randomised to support the objective of achieving a sample that is as representative of the total population as is possible
- In the construction of the survey respondent group samples, response thresholds apply to ensure the collection of sufficient data
- All items have been professionally edited for (i) grammatical accuracy and (ii) clarity and suitability of language for different audiences globally
- While there are six key stakeholder groups used for survey purposes, they are consolidated into three reporting groups to (i) ensure sufficient data for each reporting group and (ii) to avoid the potential for identifying respondents
- The survey feedback from the six respondent groups has been weighted to avoid any disproportionate influence or impact of smaller or larger groups (for example, the leadership group typically will be small in number while students or parents/carers will be much larger)
- The distribution of opinions expressed in each of the reporting groups’ feedback across the 5-point reporting scales for each sub-domain are included graphically as well as statistically in the report (reporting of standard deviations)
- The report publishes the number (or ‘count’) of respondents in each of the reporting groups
International Expert Advisory Network
The program has been strongly and generously supported by the expert advice and recommendations of the following members of the Expert Advisory Network. The Network includes the full range of expertise across school education including: policy formulation and decision making; system development and management; curriculum development and evaluation; assessment design, development and delivery; and reporting design, development and quality assurance. Highly experienced school leaders and teachers as well as former ministers of education have been included within our advisory group to ensure a broad and representative panel.
The panel therefore represents expertise ranging from the school context through district and system management, to heads of national organisations and institutions and internationally respected education experts, ministers and ministerial advisors.
|PROFESSOR NUNO CRATO OIH||Minister of Education and Science, Portugal (2011-15)|
|MR PHILLIP HELISMA||Coordinator of Student Wellbeing and Growth (7-10) Padua College, Mornington Victoria, Australia|
|DR SAJA JAMJOOM||Former General Manager of Assessment at Public Education Evaluation Commission, Saudi Arabia|
|DR PHIL LAMBERT PSM||Director, Phil Lambert Consulting|
|MR ANTHONY MACKAY AM||President and CEO, National Centre on Education and the Economy, Washington D.C.|
|MS SUSAN MANN||Former CEO Education Services Australia|
|MR GLENN MARTIN||Manager, musician and former teacher|
|MR NEIL O’SULLIVAN||Principal, Avenel Primary School, Victoria, Australia|
|MR AL-BARAA TAIBAH||Former Program Manager, Teachers Licensing, Education Evaluation Commission, Saudi Arabia|
|MS JANICE ADAMS||English language and ESL teacher|
|MR MICHAEL STEVENSON||Senior Advisor, Program for International Student Assessment, OECD Paris|
Program’s Technical Expert Advisor: Assoc Prof. Margaret Wu
The technical strength of the program is empowered by its measurement expert, Associate Professor Margaret Wu. Associate Professor Wu has an extensive background in Statistics and Mathematics. She worked as a psychometrician at the Australian Council for Educational Research for more than ten years. Between 2008 and 2011, Margaret was an associate professor at the University of Melbourne and later was an expert member of the Melbourne-based Education Measurement Services business in Australia. Margaret is a co-author of an Item Analysis software program which has been used within Australia and internationally (CONQUEST). Margaret has also written insightful and authoritative analyses of the data from the Programme for International Student Assessment (PISA) that have given content and added meaning to these highly-publicised and publicly-valued assessment results.