Appendix 3: Factor analysis

A factor analysis was conducted on a broad range of questions in the 2009 State of the Service employee survey.¹ Factor analysis is a ‘data reduction’ technique, which statistically groups together highly related questions. The factor analysis process primarily uses high correlations between question responses to group them together. A high positive correlation between two questions, for example, is where most of the responses to one question show a very similar pattern of responses as those to the other question (i.e. across individuals, high scores on one generally correspond to high scores on the other). Factors are considered to reflect underlying processes or relationships that have created the correlations among variables.²

In this case, a factor analysis was conducted to determine whether, based on the questions from the employee survey, broader groups of workplace issues or ‘factors’ existed. Therefore, each factor represents a measure of a broader construct than the individual questions in the 2009 employee survey.

This year, the factor analysis is structured around the ORIMA Research Employee Engagement Model (OREEM).

About the OREEM

The OREEM³ is based on current theory within organisational psychology and human resource management and has been validated through numerous studies conducted in Australia and overseas.⁴ This analytical framework is designed to measure the strength of relationships between the workplace factors that impact on an individual employee’s satisfaction and engagement with their job/agency and several key organisational objectives.

The main elements of the OREEM are summarised in the following figure and described below.

• A broad range of workplace factors have an impact on the level of employee engagement–measured in the model through Motivation and Discretionary Effort.⁵
• An individual employee’s engagement with their agency has a significant impact⁶ on job satisfaction, loyalty/commitment to agency⁷, productivity and broader on-the-job performance.⁸
• Loyalty/commitment to agency has a strong link with a range of organisational objectives, including agency performance, turnover and absenteeism.
  • Loyalty/commitment to agency can be conceptualised as a ‘reservoir’ of goodwill towards the organisation. Employees who are engaged and satisfied with their job tend to accumulate greater loyalty/commitment to agency over time. Conversely, disengagement and dissatisfaction tend to gradually reduce loyalty/commitment to agency.
• Improvements in agency performance and individual job satisfaction also feed back into higher levels of employee engagement.

Using the OREEM to focus analysis of the employee survey results allows prioritisation of potential improvements to those workplace factors likely to have the greatest impact on the key organisational objectives of agency performance, absenteeism and turnover.

Factors for the 2009 State of the Service employee survey

A ‘confirmatory’ factor analysis approach⁹ was used to verify the core relationships from the OREEM. Confirmatory factor analysis involves compiling a group of questions that relate to an underlying ‘latent variable’¹⁰ and running a series of statistical tests on this group of questions to ensure they form a single coherent factor. Factors are tested individually and a single question can only be allocated to one factor. The factor analysis resulted in a 14-factor structure. The 14 factors were:

• Intrinsic Rewards
• Goal Clarity
• Job-Skills Match
• Immediate Manager
• Autonomy/Empowerment
• Performance Feedback/Accountability
• Remuneration
• Recognition and Feeling Valued
• Work-Life Balance
• Team Performance and Relationships
• Career Progression
• Senior Leaders
• Agency Culture
• Learning and Development

Relationship between the factors and several key organisational objectives

Regression analysis was used to estimate relationships between the above factors within the context of the OREEM. The results of this regression analysis are summarised in the above figure, which shows that:

• four of the factors (Intrinsic Rewards, Goal Clarity, Job-Skills Match and Immediate Manager) were key drivers of Motivation/Discretionary Effort (the measure of employee engagement used in the model)
• Motivation/Discretionary Effort and seven additional factors (Autonomy/Empowerment, Performance Feedback/Accountability, Remuneration, Recognition and Feeling Valued, Work-Life Balance, Team Performance and Relationships and Career Progression) were key drivers of Job Satisfaction
• Motivation/Discretionary Effort, Job Satisfaction and three additional factors (Senior Leaders, Agency Culture and Learning and Development) were key drivers of Loyalty/Commitment to Agency.

The figure also highlights the important role of Motivation/Discretionary Effort, Loyalty/Commitment to Agency and Job Satisfaction in influencing several key organisational objectives, including:

• minimising staff turnover
• minimising absenteeism
• maximising staff productivity and overall agency performance.

Comparability with 2008 factor analysis

Results for the 2009 factor analysis are not directly comparable with the 2008 results, as although some factors may measure the same concept, the principal components of each factor are not necessarily the same. In addition, the 2009 factor analysis is based on the OREEM and resulted in a considerably different factor model to that produced in 2008.

Analysis of the survey results from 2008 suggests, however, that the 2009 factor analysis model is robust and highlights ongoing relationships between workplace factors and organisational objectives. A retrospective factor analysis of the 2008 survey results using the approach adopted in 2009 (a confirmatory factor analysis based on the OREEM analytical framework) produced a very similar model to that shown in the above figure.

Factor content

The following tables show the principal components of each factor.¹¹ In the text of the report, factor names appear with the first letter of each word capitalised to distinguish them from other uses of the word(s).

Intrinsic rewards

q17a.

I enjoy the work in my current job.

q17f.

My job gives me a feeling of personal accomplishment.

q24a.

My agency is a good place to work.

q31iib.

[How well have your expectations been met in]: ...contributing to making a difference?

q31iik.

[How well have your expectations been met in]: ...the agency’s reputation for doing important work?

Goal Clarity

q17g.

I have a clear understanding of how my own job contributes to my work team’s role.

q17h.

I clearly understand what is expected of me in this job.

q20h.

I have a clear understanding of how my work group’s role contributes to my agency’s strategic directions.

Job-Skills Match

q17d.

My job allows me to utilise my skills, knowledge and abilities.

q31iie.

[How well have your expectations been met in]: ...matching your interests/experience to the responsibilities of the job or the business of the agency?

Immediate Manager

q21d.

My manager demonstrates honesty and integrity.

q21e.

My manager works effectively and sensitively with people from diverse backgrounds.

q21h.

My manager shows concern for the welfare of his/her staff.

q21i.

My manager draws the best out of his/her staff.

q57i.

[Your workplace provides…]: Good working relationships with my manager and colleagues.

q70h.

My immediate supervisor is effective in managing people.

q21h.

My manager shows concern for the welfare of his/her staff.

q21i.

My manager draws the best out of his/her staff.

q57i.

[Your workplace provides…]: Good working relationships with my manager and colleagues.

q70h.

My immediate supervisor is effective in managing people.

Autonomy/Empowerment

q17i.

I have the authority (e.g. the necessary delegation(s), autonomy, level of responsibility) to do my job effectively.

q18e.

I receive support from my manager when I suggest new ideas.

q18f.

My agency encourages employees to examine what they do and find ways to do it better.

q21g.

My manager delegates work effectively.

q57b.

[Your workplace provides…]: A manager that encourages and manages innovation.

Performance Feedback/Accountability

q17j.

I receive adequate feedback on my performance to enable me to deliver required results.

q21b.

My manager encourages me to build the capabilities and/or skills required for new job roles.

q57c.

[Your workplace provides…]: A manager that provides effective feedback.

q57d.

[Your workplace provides…]: Realistic performance expectations.

Remuneration

q17l.

I am fairly remunerated for the work that I do.

q31iii.

[How well have your expectations been met in]: ...your remuneration package?

Recognition and Feeling Valued

q17k.

I am satisfied with the recognition I receive for doing a good job.

q20a.

The people in my work group feel they are valued for their contribution.

Work-Life Balance

q30.

Considering your work and life priorities, how satisfied are you with the work-life balance in your current job?

q70a.

My workplace culture supports people to achieve a good work/life balance.

Team Performance and Relationships

q20b.

People in my work group use time and resources effectively.

q20c.

The people in my work group cooperate to get the job done.

q20d.

The people in my work group share job knowledge with each other.

q20e.

People in my work group are honest, open and transparent in their dealings.

q20f.

People in my work group treat each other with respect.

q20g.

My work group resolves conflict quickly when it arises.

Career Progression

q17e.

My current job will help my career aspirations.

q25c.

I am satisfied with the opportunities for career progression in my current agency.

q31iia.

[How well have your expectations been met in]: … gaining experience or greater experience in the APS?

q31iig.

[How well have your expectations been met in]: ...future career opportunities in the agency?

Senior Leaders

q19c.

I feel change is managed well in my agency.

q22ia.

In my agency, the leadership is of a high quality.

q22ib.

My agency is well managed.

q22id.

In my agency, senior leaders are receptive to ideas put forward by other employees.

q22ie.

In my agency, senior leaders discuss with staff how to respond to future challenges.

q25d.

Within my agency the most senior leaders are sufficiently visible.

q42c.

Generally speaking, in my experience the most senior managers in my agency act in accordance with the APS

Values.

q70d.

Senior managers in my organisation lead by example in ethical behaviour.

Agency Culture

q24d.

Employees in my agency feel they are valued for their contribution.

q24f.

My agency operates with a high level of integrity.

q24g.

My agency deals with underperformance effectively.

q31iij.

[How well have your expectations been met in]: ...the agency’s reputation for good work practices (e.g. work-life balance, people management)?

q70b.

My input is adequately sought and considered about decisions that directly affect me.

q70c.

My organisation actively encourages ethical behaviour by all of its employees.

q70f.

I have confidence in the processes that my organisation uses to resolve employee grievances.

Learning and Development

q21a.

My manager ensures fair access to developmental opportunities for employees in my work group.

q24e.

My agency places a high priority on the learning and development of employees.

q31iih.

[How well have your expectations been met in]: ...developmental/educational opportunities in the agency?

q52i.

Overall, how satisfied are you with your own access to learning and development opportunities in your organisation?

q53.

Please rate the overall effectiveness of the learning and development you received in the last 12 months in helping you improve your performance.

q57f.

[Your workplace provides…]: Encourages increased knowledge and/or experience in the job.

q57g.

[Your workplace provides…]: Promotes access to effective learning and development.

Calculation of factor scores for each factor

A factor score was calculated for each of the 14 factors. Each factor score is the average (mean) of the responses to the questions contained in the factor. Factor scores will therefore range from 1 to 5 on a continuous scale (i.e. they will not necessarily be whole numbers). Factor scores can be used in a similar way to question results, for example, in cross- tabulations with other questions and other factor scores. Employees have one score for each factor. For ease of interpretation, the factor scores have been recoded as:

• 1 to 2.499 = agreed/satisfied
• 2.5 to 3.499 = neutral (neither agreed nor disagreed)
• 3.5 to 5 = disagreed/dissatisfied

Factor score results

The table below contains the APS-wide frequency results¹² for the 14 factors based on the approach outlined above.

Caution should be used in interpreting the proportions of ‘agreed/satisfied’, ‘neutral’ and ‘disagreed/dissatisfied’ responses for each factor shown in the table below. This is due to the sensitivity of these proportions to the cut-off points of the recoded factor scores, which may understate the proportion of positive responses for certain factors (a particular issue for factors with only two questions).¹³

Factor	Agreed/Satisfied	Neutral	Disagreed/Dissatisfied
Goal Clarity	87%	11%	2%
Team Performance and Relationships	75%	20%	5%
Immediate Manager	72%	20%	8%
Intrinsic Rewards	71%	23%	6%
Autonomy/Empowerment	64%	27%	9%
Work-Life Balance	63%	25%	13%
Performance Feedback/Accountability	57%	31%	12%
Learning and Development	56%	32%	12%
Job-Skills Match	55%	31%	14%
Agency Culture	51%	39%	10%
Remuneration	48%	34%	18%
Senior Leaders	43%	39%	18%
Recognition and Feeling Valued	40%	37%	23%
Career Progression	36%	39%	25%

Summary indexes

To complement the factor analysis, a series of summary indexes have been created to help analyse results of survey questions that (1) comprise several parts or (2) are related survey questions. The indexes operate to condense a multiple response question or a series of questions into a single index for comparative purposes. In exploring employees’ overall levels of job satisfaction, for example, a question comprising 15 attributes was summarised in a single index using a point scoring system. In this way, analysis of the 15 job satisfaction attributes can be supplemented by analysis at the summary level. These summary indexes comprised the following questions:

Motivation/Discretionary Effort (Measure of Employee Engagement)

q17b.

I am motivated to do the best possible work that I can.

q17c.

When needed, I am willing to put in the extra effort to get a job done.

Job Satisfaction¹⁴

q27a.

[Please indicate your level of satisfaction with these attributes]: Duties/expectations made clear

q27b.

[Please indicate your level of satisfaction with these attributes]: Regular feedback/recognition for effort

q27c.

[Please indicate your level of satisfaction with these attributes]: Chance to be creative/innovative

q27d.

[Please indicate your level of satisfaction with these attributes]: Chance to make a useful contribution to society

q27e.

[Please indicate your level of satisfaction with these attributes]: Seeing tangible results from my work

q27f.

[Please indicate your level of satisfaction with these attributes]: Opportunities to utilise my skills

q27g.

[Please indicate your level of satisfaction with these attributes]: Opportunities to develop my skills

q27h.

[Please indicate your level of satisfaction with these attributes]: Good working relationships

q27i.

[Please indicate your level of satisfaction with these attributes]: Appropriate workload

q27j.

[Please indicate your level of satisfaction with these attributes]: Salary

q27k.

[Please indicate your level of satisfaction with these attributes]: Opportunities for career development

q27l.

[Please indicate your level of satisfaction with these attributes]: Interesting work provided

q27m.

[Please indicate your level of satisfaction with these attributes]: Flexible working arrangements

q27n.

[Please indicate your level of satisfaction with these attributes]: Good manager

q27o.

[Please indicate your level of satisfaction with these attributes]: Appropriate level of autonomy in my job

Loyalty/Commitment to Agency

q25a.

I am proud to work in my current agency.

q25b.

I would recommend my current agency as a good place to work.

Loyalty/Commitment to APS

q26f.

I am proud to work in the Australian Public Service.

q26g.

I would recommend the Australian Public Service as a good place to work.

The scores for Motivation/Discretionary Effort, Loyalty/Commitment to Agency and Loyalty/Commitment to APS were calculated using the same scale as that used for the factors. The following table shows the APS-wide frequency results for these summary indexes.

Index	Agreed/Satisfied	Neutral	Disagreed/Dissatisfied
Motivation/Discretionary Effort	87%	11%	2%
Loyalty/Commitment to Agency	65%	23%	11%
Loyalty/Commitment to APS	80%	16%	4%

The score for Job Satisfaction was calculated using a 10-point scale based on the following approach:

• 9 to 10 = high satisfaction
• 6 to 8 = moderately high satisfaction
• 3 to 5 = moderately low satisfaction
• 0 to 2 = low satisfaction

The table below shows the APS-wide frequency results for Job Satisfaction using this approach.

Index	9 to 10	6 to 8	3 to 5	0 to 2
Job Satisfaction	26%	55%	16%	3%

NEXT » State of the Service survey methodologies