Chinese Personality and Performance at Work Research Research Results

The focus of this research program was to assess the utility of Western and indigenous models and measures of personality in Chinese and Australian organisations. Data relating to psychometric properties, factor structure, construct and criterion-related validity was collected for the 15FQ+ and the CPAI-2 in Hong Kong SAR, mainland China and Australia in order to address this aim. Additionally, in order to compare factor structure cross-culturally, 15FQ+ test data was acquired from Australia, New Zealand and the United Kingdom.

9.2 Findings in relation to the major aim

The findings of the series of studies suggest that whilst there are some issues surrounding the internal consistency of a number of the 15FQ+ scales (particularly in mainland China), the translated 15FQ+ does have utility as a measure of personality and as a predictive tool in China. Stronger confidence for this statement was found in Hong Kong than in mainland China given the higher quality of the item/scale/test analyses and the larger sample size. The CPAI-2 indigenous scales generally were found to possess less than acceptable psychometric properties. Although significant correlations were found between the CPAI-2 indigenous scales and performance variables, the CPAI-2 did not add any incremental validity to the 15FQ+ in the prediction of performance in China. In Australia, the CPAI-2 indigenous scales were again found to have generally unacceptable levels of internal consistency. However, it was with an Australian sample that the first evidence of incremental predictive utility of the CPAI-2 indigenous scales, above the 15FQ+, was found. The joint factor analyses produced different results in Australia compared to Hong Kong. In Australia, the CPAI-2 indigenous scales formed a separate factor in the five-factor solution and two separate factors in the six-factor solution. These separate factors were not loaded upon by any of the 15FQ+ scales. This had the effect of reducing the 15FQ+ scales into 4 factors with scales postulated to be from global factors A and O appearing to load on the same factor. These loadings and factor structures were not supported with the Hong Kong data. Instead, a five-factor solution provided the best fit and the CPAI-2 indigenous scales loaded on the 15FQ+ Emotional Stability (all non-Interpersonal-relatedness scales) and Conscientiousness factors (Interpersonal-relatedness scales). Outside of its country of development, it appears that the CPAI-2 indigenous scales do have utility and measure something distinct from that which is measured by Western-developed personality tests. However, why this was not the case in Hong Kong remains unresolved and in need of future research.

9.3 Implications of the findings

9.3.1 Psychometric implications

The data indicates that Western personality tests that have been translated into Chinese can have acceptable levels of internal consistency. Although still indicating a need for further refinement, these can be higher than the levels of internal consistency achieved by tests developed with Chinese samples. Clearly, the reliability and validity of such tests is dependent, in part, upon both the test’s construction and the quality of its translation. There were however obvious differences in the psychometric properties of the Hong Kong SAR 15FQ+ data in comparison with the mainland China data. This may imply that the Western model is less reliable on the mainland, or the finding may revolve around the homogenous sample that was used.

The full version of the CPAI-2 attempts to measure both ‘universal’ and indigenous dimensions of personality. It is possible that this attempt to provide an omnibus measure of cross-cultural personality is too demanding. With the 103 indigenous items alone already exceeding the length that some organisations are willing to accept in administration time, it may be advisable to provide the CPAI-2 as simply a measure of the indigenous aspects of personality and to focus on increasing the quality of measurement of its items. The measurement of the ‘universal’ aspects of personality is perhaps done with greater internal consistency through the use of the longer established and internationally critiqued Western-developed models and measures. This does not imply that the indigenous factors are not important in China or elsewhere, although further organisation-based research is required in order to assess the contribution of these scales to the prediction of performance. (It is noteworthy that, although the CPAI-2 was not developed specifically as a measure of personality to be used in the workplace, it is however currently used in Chinese workplaces and with a seeming lack of awareness of its poor measurement properties.)

This research program took the further investigative step of in-depth item/scale/test quality analyses. It was seen that an alpha coefficient can, on its own, misleadingly suggest that a scale has good properties. However, an assessment of how items that are supposed to load on other scales in the test actually load on a non-keyed scale, for example, provides an otherwise overlooked assessment of measurement noise. There is difficulty in comparing the findings of these analyses with non-existent previous analyses in this area. In view of the additional quality information that can be gleaned from such analyses, it is suggested that any future research that aims to assess the psychometric properties of a test does not stop at internal consistency reliability, but considers the use of these more detailed analyses. This will ultimately ensure that measurement scales measure the attribute that they purport to measure in a ‘pure’ sense and without measurement noise from other scale’s items.

9.3.2 Organisational implications

From an organisational perspective the research findings suggest that in both Hong Kong SAR and mainland China, Western-developed tests based on the Five-factor Model and Cattell’s model can be used with clear utility. However, continued research is required given that the current research was at the infancy of personality-performance research in China. Furthermore, where these tests are used, attention needs to be paid to their psychometric properties – to the extent of examining those properties after each group administration. Where issues concerning lack of reliability occur, careful interpretation of the offending scale is advised. At the current time, and based on the current program of research, the findings in relation to the CPAI-2 indigenous scales suggest that in Hong Kong SAR and mainland China, there is questionable utility in the use of these scales. The Chinese data shows these scales as loading on Emotional Stability and Conscientiousness -- factors that are already measured by the 15FQ+. Assessment of the incremental predictive utility of the CPAI-2 indigenous scales over and above the 15FQ+ in China did not reveal any significant findings. Interestingly, in Australia, there was some support for incremental predictive utility with the HAR scale being associated with a course grade. However, poor reliability demands a cautious interpretation over future CPAI-2 test administrations in Australia. Furthermore, given that this finding occurred with a student group within which a small number of participants agreed to providing their grades, it is unlikely that this finding will extrapolate to work organisations. The finding that the CPAI-2 indigenous scales formed a unique factor in a joint factor analysis in Australia needs to be supported by further research. If the CPAI-2 indigenous scales continue to load uniquely, suggesting a separate indigenous factor, organisation-based research is required in order to establish the predictive utility of this factor and the scales which compose it.

9.3.3 Implications for personality, performance, and predictive models

The discussion sections of each study have noted the degree to which the findings have supported previous research that has attempted to relate personality with performance. The main position is that there is a relationship between personality, as measured by a Western-developed test, and performance both in the West and in China. Predictive models have demonstrated that a significant and high percentage of the variance in performance can be accounted for by these personality traits. For all but one of the studies (Study 6), this was found for the 15FQ+ scales, but not for the indigenous scales of the CPAI-2. However, a large percentage of the variance in performance nonetheless remains unaccounted for. This variance is likely to be attributed to other aspects of the individual such as previous experience, educational attainment and general ability. In China, just as in the West then, although personality can be implicated in the prediction of performance at work (as well as academic performance), decisions regarding individuals should not be based on personality test results alone. Other factors that contribute to the total relationship between the individual and performance must be investigated and also be considered.

An important finding from the present program of research has been the utility that the narrow-band primary traits of the 15FQ+ have in providing a fuller, yet more refined, explanation of personality-performance relationships. At times, however, the relationship between a single, narrow-band trait and performance has not been substantive enough to produce statistically significant results. However, the composite global factor has, on some of these occasions, highlighted significant relationships between personality and performance. The implication is that, wherever possible, both narrow- and broad-band assessment of personality should be carried out. Users of tools such as the NEO-PI or FACET 5 should take particular note of this point.

9.3.4 Generalisability of the results

It is suggested that in China, the results of this study are generalisable to work organisations and university students in Hong Kong SAR given the wide range of organisations and student groups from which the participants came. The mainland China dataset came from one, mostly male, ship crew sample and thus generalisation is not advised. The results of the Australian research may be generalisable to students, as well as administrative, nursing and management staff having used two student groups and two general hospital employee groups.

9.4 Limitations and future research

The two major limitations within the program of research relate to both sample size and the internal consistency of the measures used.

There are well-known difficulties involved in finding organisations to participate in research programs that are not directly linked to their current organisational objectives. Furthermore, where invitations to participate in research may, by implication, lay question upon the reliability, validity and utility of the current process of selection and or development, there is even less chance of receiving positive organisational support. Carrying out research with organisational participants not only costs the organisation in terms of employee time, but may create an atmosphere wherein change is expected and this too can be of concern to the decision-makers. It is for these reasons that despite approaching over 150 organisations for their support in this research, only 7 were forthcoming.

For those organisations that agreed to participate, there was the necessity to engage the employees as participants. Again, this can be a difficult prospect. Having overcome this barrier, there are then idiosyncrasies with each organisation which may result in a reduction of valid data. In the current research program, this was noted with Organisation 1 in Study 4, wherein two different performance appraisal systems had been used and this had the effect of dramatically reducing the data available for the personality/performance correlations and regressions. Suggestions have been made throughout relevant discussion sections in relation to how the sample size may be increased and where caution should be exercised in interpretation of findings as a result of the low sample sizes and the consequent reduced statistical power. Sample size issues have effectively reduced the strength of the some of the relationships that are existing within the data and, in turn reduced the amount of confidence that can be placed in the findings.

9.4.2 Reliability of the personality scales

Given the poor reliability of many of the indigenous scales of the CPAI-2 and the generally low reliability that has been reported by other researchers (e.g., Cheung, 2004), there is an obvious need for further work on the items within this questionnaire. The 15FQ+ also returned unacceptable reliability on a number of items, especially with the mainland China participants and thus further refinement is suggested here. This refinement has been put in place by the publisher (L.Paltiel, personal communication, 6 November, 2004). The major issue is that many of the assumptions and interpretations throughout the thesis rely on the instruments concerned having good psychometric properties. Until greater levels of internal consistency are reported for both the CPAI-2 and the 15FQ+, there must be caution in the interpretation of the findings within this thesis, as well as the interpretation of individual test results and the association with performance at work that may be afforded to them.

9.4.3 Reliability and validity of the outcome measures

A final issue of critical importance throughout the thesis is that of the reliability and validity of the outcome measures with particular reference to the performance variables in Study 4 and Study 5. There is a tendency to assume that ‘objective’ performance appraisal ratings are reliable and valid. However, the only scientific way to confirm this within a participating organisation is to carry out a study whereby independent raters make an assessment of the candidate’s performance, based on (covert) observation and/or assessment of a specially-designed task or assessment/development centre and correlate these independent ratings with those from the supervisor. Even so, this does not address the reliability and validity of performance appraisals made over time, such as with the Study 5 data. Future studies may validate the performance appraisal process within organisations prior to carrying out similar research to that described within this thesis. However, as with the sample size issue mentioned above, there will likely be resistance to participation from organisations due to the fact that negative results will have the effect of invalidating the current appraisal process. For the studies within this thesis, it is suggested that the appraisal systems were as reliable and valid as most similar organisations, especially based on the patterns of observed correlations between performance variables and personality traits.

The 15FQ+ has been shown to have good psychometric properties when administered in its English language form outside of its country of development (Australia and Hong Kong: Study 1). Likewise, the 15FQ+ was found to have good psychometric properties and to correlate with, and be implicated in prediction models of, well-being, coping styles and self-appraised performance with Australian employees (Study 2). Following translation to Traditional Chinese (Study 3), the 15FQ+ was found to have reasonable psychometric properties that would benefit from some refinement to the translated questionnaire. However, at this stage, 15FQ+ scales did nonetheless correlate with NEO-FFI scales measuring similar personality characteristics and a significant relationship was found between Conscientiousness and GPA for Hong Kong Students. This scale was also implicated in a predictive model of personality upon GPA. Following refinements to the translation, a second administration of the Traditional Chinese 15FQ+ was carried out with Hong Kong professionals (Study 4). The Traditional Chinese CPAI-2 indigenous scales were also administered. The 15FQ+ was found to have better, although still not altogether acceptable, psychometric properties than in Study 3. The psychometric properties of the Traditional Chinese CPAI-2 indigenous scales were mainly not acceptable. Significant relationships were found between personality (15FQ+ and CPAI-2) and performance in this study and 4 significant models were determined in the prediction of performance based on personality. The CPAI-2 did not add any significant incremental validity over and above the 15FQ+ in the prediction of performance.

A joint factor analysis of the Traditional Chinese 15FQ+ with the Traditional Chinese CPAI-2 indigenous scales revealed that the CPAI-2 scales did not form a unique factor. Comparison of the 15FQ+ factor structure for the Hong Kong data with a United Kingdom normative reference sample revealed that the datasets were highly congruent. Data from mainland Chinese participants (Study 5) revealed less quality in the psychometric properties for the Simplified Chinese 15FQ+ and Simplified Chinese CPAI than had been observed in Study 4. However, a number of significant correlations were observed between personality and performance for both the 15FQ+ and the CPAI-2. In regression models predicting performance from personality, the CPAI-2 indigenous scales had no incremental predictive utility over and above the 15FQ+. In contrast to the results of Study 4, comparison of the 15FQ+ factor structure for the mainland China data with a United Kingdom normative reference set revealed that global factors A, E and O were not congruent, while C and N were. In the final study (Study 6), the English version of the 15FQ+ was found to have good psychometric properties, while, in the main, the English version of the CPAI-2 showed unacceptable properties. Personality was found to correlate significantly with student grades. However, in a significant regression model predicting one of these course grades from personality, the CPAI-2 was found to have incremental validity over and above the 15FQ+, with HAR being the significant predictor. Joint factor analysis of the English language 15FQ+ and CPAI-2 indigenous scales revealed a unique factor for the CPAI-2 within a five-factor solution or two unique factors within a six-factor solution. The 15FQ+ factors were effectively reduced from the usual 5 factors to 4 factors by the merging of aspects of global A with global O.

The 15FQ+ does have utility in a Chinese setting, but further work is required in order to address issues of low reliability for a number of scales. The CPAI-2 indigenous scales, whilst seeming to have little utility in a Chinese setting, appear also to have utility in a non-Chinese setting. However, further work is again required with a larger sample in order to make these findings generalisable and, similarly to the 15FQ+, further work is required to increase the reliability of this instrument.

A call must be made for continued research in this new area in order to discover whether the indigenous factor(s) is/are unique in both China and the West and, if it is, whether it offers anything in terms of explanatory power in personality theory and more importantly, in terms of performance prediction. Data from the current research program has pointed to the Western-developed test as the test of choice in the West and in China. Future research will provide confirmation or otherwise of this claim

Research Results - A synopsis

No reproduction without permission.
This study forms a major part of the original research in conjunction with the University of Queensland, Australia and in conjunction with Dr. Peter Newcombe at that University and Professor Paul Barrett who was at the University of Auckland at the time.

Research Results - A synopsis

No reproduction without permission. This study forms a major part of the original research in conjunction with the University of Queensland, Australia and in conjunction with Dr. Peter Newcombe at that University and Professor Paul Barrett who was at the University of Auckland at the time.

No reproduction without permission.
This study forms a major part of the original research in conjunction with the University of Queensland, Australia and in conjunction with Dr. Peter Newcombe at that University and Professor Paul Barrett who was at the University of Auckland at the time.