personality.cn The Chinese Personality
at Work Research Project
University of Queensland, Australia, Dr. Graham Tyler & PsyAsia
International
Apollo Profile Research Results - A synopsis
No reproduction without permission.
NB: This research study was not part of the
original University of Queensland Research. It does however form
part of the Chinese Personality at Work Research Project
and was conducted in association with the
Apollonean Institute and Professor Richard Hicks at Bond University,
Queensland, Australia.
Outline of the study
In October 2007, PsyAsia International
in Hong Kong and the Apollonean Institute in Australia embarked
on a research study with students from the Hong Kong Institute for
Vocational Education (HKIVE). A traditional (and simplified) Chinese
version of the Apollo Profile already existed by way of the publisher
producing it for a large client job some time ago. With a plan to
develop the Apollo market in Asia, all parties were interested in
conducting a reliability and validity study on Hong Kong Chinese
people. The project was headed in Hong Kong by Dr. Graham Tyler
who has a PhD based on psychometric assessment and validating tools
for predicting performance at work in Asia. Mr. Jim Bowden, publisher
of the Apollo Profile headed the project from Australia with analytical
and statistical support from Professor Richard Hicks at Bond University
Queensland.
Phase One of the research required
participants to complete an online version of the Traditional Chinese
version of Apollo. One month later, Phase Two of the study required
the same individuals to complete the same version of the Apollo
again (to assess test-retest reliability). Analyses were undertaken
by Professor Hicks using data from those who undertook both Phases
of testing - a total of 262 Hong Kong Chinese people.
The final phase of the research will
compare the student's personality profiles with their end of year
exam result (GPA).
Study Results (text reproduced from
Professor Hick's report with minimal editing)
The full profile correspondence of
total scale scores at times 1 and 2 was very good overall: test-retest
pattern of scale scores being an averaged 0.96 (Pearson Product
Moment coefficients across the 34 scale scores). The range was .80
to .99 (SD of 0.024). The conclusions drawn using the full Apollo
Profile and its combined scales would have been similar at times
1 and 2.
The Apollo Profile is meant to be
used as a whole (that is, combinations of scales or profiles) and
no one scale should be used in applications in practice. Therefore
the overall patterns of scores used in practice with this Chinese
sample (to predict suitability) would have yielded similar results
at times 1 and 2. So test-retest reliability is present for the
profile pattern.
However, it is also usual to
identify how well the scales and sub-scales of questionnaires operate
if they were to be used ‘on their own’-- and test-retest
and internal Alpha coefficients are usually used for this purpose.
There is much confusion about the most desirable levels for this
latter approach (internal Cronbach Alpha coefficients) and this
aspect is addressed later.
The
test-retest coefficients by scales, for the Chinese language version,
showed a range from .53 to .78 (for the 34 main scales taken individually
over the 262 student responses at times 1 and 2: mean: 0.65; SD
.08). These are on a scale for scale basis satisfactory though somewhat
lower than the coefficients obtained in the English-language version,
and much lower than the overall pattern result of average 0.96.
The implication is that while the scores on individual scales change
somewhat when considered alone, when the scales are considered in
concert there is, nevertheless, considerable reproducibility.
The
internal consistency figures (alpha coefficients) are generally
satisfactory but several scales as shown in the tables need further
attention. In the Chinese student sample (on the first administration)
17 of the 28 internal alpha coefficients for the main scales were
above .60, 7 scales were between .40 and .59 (on first measure)
but 4 of the 28 scales fell outside this range. The major scales
appeared to work soundly - these were the scales assessing Agreeableness,
Conscientiousness, Extraversion, Innovation-Openness, and Well-being-Resilience
(‘Stress Resilience’)- all similar to the Big Five;
and Teamwork, Power, Remuneration, Security, Self-organisation,
Conformity, Analysing- Analytical, Recognition, Intimacy, Collaboration,
Responsibility and Goal-setting.
There
were several items (14) identified as causing problems for some
of the scales in their alpha coefficients in the Chinese sample;
that is, their ‘deletion’ would increase the obtained
alpha coefficients. It could be that there are translation or cultural
difficulties with these items.
The Pearson PM
r between the alpha coefficients (4070) &Chinese test-retest
1-2 coeffs was .70. The ‘order’ is much the same in
each case. Those scales that are good or bad in terms of internal
alpha consistencies are also respectively good or bad on test-retest.
The Pearson PM r between the alpha
coefficients (4070) &Chinese time 1 alpha coeffs was .80 (and
.72 at Time 2). The ‘order’ is much the same in each
case, but suggests a dropping off at time 2.
The Pearson PM r between the alpha
coefficients at Chinese times 1 and 2 is a strong .92, suggesting
similar patterns of consistency across the two testings.
The PPMr Chinese T1 and the external
Test-Retest coefficient was .72; but between Time 2 alphas and the
Test-Retest coefficients just .60. The ‘order’ is somewhat
different for the time 2 measure- maybe tiredness or rushing applied
to time 2? The drop is consistent at T2 with the external ‘4070’
measure with a similar drop-off found. .
The Time 1 and time 2 order
of alpha coefficients was very similar (.92) suggesting despite
any ‘drop-off effects’ that similar responses overall
were being given on the two occasions. This was supported by the
average correlation coefficient of the 262 paired patterns/profiles
(of 34 scales) being .964. In general, advice given at T1 would
have been similar to the advice given at T2, when profiles of scores
and not any individual scales are used. It is not good practice
of course in any case to base decisions on individual scales alone
and no effective practitioner would do this. The patterns of the
scales are what matters. (The use of the Suitability scales follows
this practice of using the main profiles).
Exploratory factor analysis
An indicative exploratory factor analysis
(varimax in this trial) of the 28 scales not including the six sub-scales/facets
for the Chinese sample (of only 262) suggested tentatively that
there were five main underlying factors in the responses similar
to the factors on the ‘western’ / international samples
(with Innovation = Openness, Extraversion, Agreeableness and Conscientiousness
factors highlighted, but with a Competitive-conforming factor as
5th, rather than a ‘resilience’ or negative neuroticism
factor). These differences need further study. The first five factors
were :
Factor 3: Competitive (.82), Conforming-pragmatic
(.76), Remuneration (.59) and Power (.54) with Directive and Compromising
but negative Teamwork and Agreeableness (.40s)
Factor 4: Ambition (.54), (negative) Agreeable (-.65),
Achievement (.54), Decisive (.50), with Proactive, Self-organisation
and Responsible (each .44) and Goal-setting (.38)
Of course these results are indicative only
and are based on the first testing. Further studies will be
needed to confirm the structures.
Predictive Validity Results
Currently under analysis. Please check back later.
CAVEAT OVER THE FULL STUDY
There are problems with using a student
sample, as a means of confirming alpha reliability coefficients
for a questionnaire that is aimed mainly at people in the workplace
or about to join the workplace. Further while the test-retest coefficients
were on the whole reasonable, I have a concern about the integrity
of the second sample. There was some evidence that commitment/ interest/
motivation may have fallen off in the second testing. We need further
Chinese studies- both of people in the workplace and of Chinese
university students.
