Output 4: Evaluation

Output 4.2: Research, analysis and evaluation

The Commission undertakes research and evaluation that contributes to a confident, high quality and sustainable APS. The performance of the APS is evaluated through the Commissioner’s State of the Service Report to Parliament, and through more specific issuesbased evaluation projects.

Output 4.2.1: Evaluate and provide information on the performance of the APS through the State of the Service Report (including Workplace Diversity Report) and Statistical Bulletin

Under s.44(2) of the PS Act, the Public Service Commissioner is required to report annually on the state of the APS. While the Act does not specify the issues to be covered in the report, the Commissioner’s approach has been to focus evaluation on the extent to which agencies incorporate and uphold the APS Values (in line with s.41(1)(a) of the PS Act).

The State of the Service Report 2002—03 was closely structured around the four groups of APS Values, with sections covering relations with the Government and Parliament, relations with the public, workplace relationships and personal behaviour. Separate chapters covered workplace diversity and building APS capability. Key findings are outlined on pg 85.

In previous years, the State of the Service Report and the Workplace Diversity Report were tabled separately as part of the State of the Service Series. In 2003 for the first time the Workplace Diversity Report was included as a chapter in the State of the Service Report 2002—03 and this practice will continue in future reports.

The Commission’s 2003—04 Corporate Plan identified strengthening the evidence base in the State of the Service Report as a priority activity.

To develop the State of the Service Report 2002—03, information was gathered from the Commission’s own research and databases (particularly APSED), published and unpublished material from other agencies (particularly central agencies) and Parliamentary Committee and Australian National Audit Office (ANAO) reports. A survey of all agencies that had 20 or more APS employees provided information on a large range of management and capability issues. The agency survey was conducted on-line in 2003 (previously it had been a paper survey) with a 100% response rate from the 89 agencies.

This year saw the introduction of a further source of information: the initial annual employee survey, sent to almost 6,500 randomly selected APS employees. The employee survey was an important initiative and provided data on employee attitudes to, and understanding of, a range of issues including the Values, Code of Conduct, merit, work/life balance, job satisfaction, service to the public, relations with the Minister and the Parliament, diversity, learning and development. As many questions were similar in both the agency and employee surveys, the employee survey was in part a ‘reality check’ on responses to the agency survey. The employee survey had a 51% response rate– a creditable result for a voluntary survey. Large agencies (i.e. those with over 1000 APS employees) received a summary of their own employees’ responses to the employee survey where they were statistically valid. Small and medium sized agencies received a benchmark summary of all small or medium sized agencies respectively. The results of the employee survey were collated in aggregate form and were the basis of a publication, State of the Service Employee Survey Results 2002—03.

Following the approval of the Joint Committee of Public Accounts and Audit and the Secretary of the Department of the Prime Minister and Cabinet to table the State of the Service Report by the end of November each year, the State of the Service Report 2002—03 was tabled in Parliament on 27 November 2003.

The report was distributed in hard copy to APS Agency Heads, Corporate Heads of Management, all SES, Agency Contact Officers for surveys, the HR Directors Network, Diversity Co-ordinators, State/Territory/NZ Public Service Commissioners, selected non-APS Agency Heads, relevant academics and overseas contacts. The State of the Service Employee Survey Results 2002—03 was distributed in hard copy to APS Agency Heads, Corporate Heads of Management, Agency Contact Officers and State/ Territory/NZ PS Commissioners. Both publications are available on the Commission’s website.

The report also highlighted four critical challenges for the APS in the coming year:

ensuring there is a strong pool available to take on leadership roles into the future
actively addressing diversity, both to improve capability and performance, and to overcome concerns particularly about Indigenous Australians, people with a disability and people from non-English-speaking backgrounds
getting more from organisations with respect to workforce planning, performance management and more structured learning and development
ensuring an effective relationship between the APS and the Government and Parliament.

The State of the Service report for 2003—04 will again include the results of an employee survey completed on-line by most of those invited to participate, with paper copies available to employees without access to online facilities. It will also draw on the results of the annual agency survey. The employee and agency surveys have been designed so that time series data will be available from year to year, enabling better coverage of trends in the APS.

APS Statistical Bulletin

The APS Statistical Bulletin 2002—03 provided a statistical overview of APS staff employed under the PS Act, at June 2003, and of staff movements during the 2002—03 financial year. The Bulletin also included summary data for the past ten years. Its format generally followed that of the previous year, with the addition of tables showing:

the retention of ongoing staff in each agency between June 2002 and June 2003
operative status by agency and gender
promotions by classification, age group and gender
transfers by classification, age group and gender
separations by age group and gender.

Some agency tables also included more detail comparing June 2002 and June 2003, or the 2001—02 and 2002—03 financial years. This was possible because there were very few changes in Administrative Arrangement Orders during that period.

The 2002—03 Bulletin was distributed to all APS Agency Heads, members of the Corporate Management Network (COMNET), State, Territory and New Zealand Public Service Commissioners, and major public and university libraries. It is available both in hard copy and electronically on the Commission’s Internet site. The increasingly popular electronic version enables users to download electronic versions of tables without the need for manually entering data. This year’s electronic version had an improved layout which helped users to navigate more easily between tables.

Agencies use the Bulletin data to benchmark themselves against APS-wide trends, and for workforce planning.

TABLE 21: Performance information–Output Component 4.2.1

Target	Result
Quantity Annual State of the Service Report (including the Workplace Diversity Report) and Statistical Bulletin.	Achieved–produced State of the Service Report 2002—03 (including Workplace Diversity Report), Statistical Bulletin 2002—03 and State of the Service Employee Survey Results 2002—03.
Quality 100% compliance with tabling requirements for Government. Accurate and useful reports to Government and Parliament on the performance of the APS.	Achieved–State of the Service Report (SOSR) complied with tabling requirements for Government. Reports produced with high standards of accuracy. Positive feedback received on the State of the Service 2002—03 from senior public servants and Agency Contact Officers. The report was also positively noted in the IPAA report and The Canberra Times.

Output 4.2.2: Specific-issue evaluation program

Another element of the Commission’s strategic priority relating to evaluation and assurance for 2003—04 was the introduction of a program of specific-issue evaluation projects. The aim is to conduct one or two of these projects a year.

The evaluation program for 2003—04 was finalised in October 2003 after the results of the agency and employee surveys for the 2002—03 State of the Service Report had been analysed. The analysis of the survey results helped to prioritise issues for inclusion in the evaluation program.

The major project in the program was an evaluation of agency management of suspected breaches of the Code of Conduct. Work on this project commenced in December 2003. It was decided to adopt essentially a case study approach of six APS agencies. After discussing the proposed methodology with the six agencies, interviews with relevant staff and the facilitation of focus groups of employees commenced in April 2003 and will finish for all six agencies by mid- July 2004. The results of the evaluation will inform the 2003—04 State of the Service Report. In addition, a good practice guide on the management of suspected breaches of the Code of Conduct in APS agencies will be released by the Commission in the first quarter of 2005.

The Evaluation Program also included a number of other projects:

an evaluation of agencies’ guidance on interactions with ministerial offices. This project examined in detail the factors influencing employees’ confidence in balancing the APS Values in their interactions with ministers’ offices, and considered what steps agencies could take to boost that confidence. The results will feed into both the State of the Service Report for 2003—04, and a good practice guide for use by agencies to be released in late 2004
a survey of participants at relevant APS Commission training courses aimed at exploring how perceptions of merit in selection processes are affected by an employee’s knowledge of the actual merit requirements in the regulatory framework. The survey results may feed into an evaluation project of how merit is applied in selection processes subject to consideration of priorities for the Commission’s 2004—05 Evaluation Program
a review of APS agencies’ written workplace diversity programs. The focus of this project is to find out whether agencies are complying with the statutory requirement to have such written programs and the quality of existing programs. The results will feed into the 2003—04 State of the Service Report
a number of small scale, in-depth reviews of results from the 2002—03 employee and agency surveys for the State of the Service Report. These have resulted in internal working papers designed to contribute to policy development within the Commission.

The 2004—05 Evaluation Program will be finalised later in 2004 when the results of the 2003—04 surveys for the State of the Service Report have been analysed and can inform evaluation priorities.

TABLE 22: Performance information–Output Component 4.2.2

Target	Result
Quantity At least one new evaluation completed aimed at improving administration and service delivery in departments and agencies.	Achieved–the evaluation stage of a project to deliver a good practice guide on agencies’ interactions with Ministers’ offices has been completed; the guide is in preparation. Two evaluations are partially complete: • major evaluation of APS agencies’ management of suspected breaches of the Code of Conduct • a review of APS agencies’ written workplace diversity programs
Quality High level of satisfaction of Steering Committee that evaluations will influence future performance.	Achieved–group for the evaluation into suspected breaches of the Code of Conduct met in March 2004 and expressed satisfaction with proposed methodology. Reference Group will meet in first quarter of 2004—05 to discuss findings from the evaluation and material for good practice guide.

Output 4.2.3: APS employment database and related research

The Commission maintains the APS Employment Database (APSED), which is the central database on APS employment. In addition to being an important source of information on overall APS workforce trends, APSED is an important source of information for cross-service accountability, particularly for the State of the Service Report. APSED is also used as the sampling frame for the State of the Service Employee Survey. Improving data sets, such as APSED, was part of the Commission’s strategic priority relating to evaluations and assurance for 2003—04.

APS Employment Database

In the past year, the provision of data by agencies has improved considerably. Almost all agencies are now providing data regularly, and through automated processes. The Commission continues to work with the vendors of HR systems and agencies to ensure that APSED requirements are incorporated into updates to HR systems. In addition, the Commission works closely with agencies to ensure the integrity of data sent by agencies.

The provision of diversity data relating to people with a disability, race or ethnicity and Indigenous status continues to be a concern. In December 2003, the Commissioner wrote to the heads of agencies with more than 1000 staff, advising them of the quality of their employee provided data, and asking for their assistance in ensuring that resources are applied to fixing this problem. Implementing strategies to improve the quality of this data will require long-term and incremental approaches to achieving significant improvements.

Trends in engagements paper

Work on a research paper analysing trends in engagements to the APS over the past twenty years has been substantially progressed. The paper will be completed in the first half of 2004—05. The paper includes information about changing patterns of engagement over the study period in the following areas:

classification
age
EEO status
educational qualifications
employment category and status
re-engagements.

Implementing Organisational Renewal

In response to the demographic challenges identified in the MAC 3–Organisational Renewal report, the Commission in consultation with relevant agencies developed a package of materials designed to assist agencies to adapt their human resource management strategies to retain and attract mature-aged employees. Drawing in part on APSED data, the package includes guidance on leadership for mature-aged workers and materials addressing the challenge to provide flexible working arrangements for older workers. The package, Implementing Organisational Renewal: Mature Aged Workers in the APS, was launched in December 2003, by the Hon. Kevin Andrews, the Minister assisting the Prime Minister for the Public Service.

Almost 2000 copies of the package were distributed to APS agencies, and, due to ongoing demand, the Commission has funded a reprint. The package of materials is also available on the Commission’s website. Regional launches have also been used to promote the package and to maintain the focus on the challenges of workforce ageing.

Price for Output 4

The total price for Output 4 was $2.488m.

This is slightly higher than the estimated price of $2.062m published in the Additional Estimates in February 2004.

TABLE 23: Performance information–Output Component 4.2.3

Target

Result

Quantity
Number of publications produced–Estimate 3

Significant progress–the publication Implementing Organisational Renewal: Mature Aged Workers in the APS was released in December.

The Trends in Engagements research paper was substantially progressed at end June 2004, and will be completed in the first half of 2004—05.

Quality
Reliable and comprehensive information on the composition of the APS in a useful format

Achieved–high quality and timely data has been made available and used in a number of publications including the State of the Service Report, the Commission’s Indigenous Employment Project, Implementing Organisational Renewal: Mature Aged Workers in the APS and various Ministerial and Commission briefings on particular issues.

Initial work with agencies to improve data quality has been undertaken.

An APSED Internet Interface has been developed.

This page is available from www.apsc.gov.au/annualreports/0304/output42.htm
For information and help with this site go to Using our site