Yan Liu, Edward L. Korn, Hee Soo Oh, Helmer Pearson, Tian-Min Xu, and Sheldon Baumrindf
Beijing, China, Rockville, Md, San Francisco, Calif, and Newark, NJ
Introduction: This study continues our assessment of agreement and disagreement among 25 Chinese and
20 US orthodontists in the ranking for facial attractiveness of end-of-treatment photographs of randomly sam-
pled growing Chinese and white orthodontic patients. The main aims of this article were to (1) measure the
overall pattern of agreement between the mean rankings of US and Chinese orthodontists, and (2) measure
the strength of agreement between the rankings of the US and Chinese orthodontists for each patient.
Methods: Each judge independently ranked standard clinical sets of profile, frontal, and frontal-smiling pho-
tographs of 43 US patients and 48 Chinese patients. For each patient, a separate mean rank was computed
from the responses of each group of judges. Pearson correlations between the mean ranks of the 2 groups of
judges were used to measure their overall agreement. Paired and unpaired t tests were used to measure the
agreement between the judges of the 2 groups for each patient. Results: The overall agreement between the
mean rankings of the US and Chinese judges was very high. For the US patients, the correlation between
the Chinese and US judges means was r 5 0.92, P\0.0001. For the Chinese patients, the analogous value
was r 5 0.86, P\0.0001. Agreement between the 2 groups of judges concerning each patient was also gen-
erally strong. For two thirds of the patients, the mean ranks of the US and Chinese judges differed by less than
1 unit in a scale of 12. However, for 6 patients considered individually (5 Chinese and 1 US), the assessment of
the 2 groups of judges was statistically significantly different at P values ranging from 0.02 to less than 0.0001,
even after the Bonferroni correction. Conclusions: These findings demonstrate that orthodontic clinicians can
reliably identify and rank subtle differences between patients, and that differences between judges and be-
tween patients can be distinguished at a high level of statistical significance, given appropriate study designs.
However, the reasons clinicians give for the differences in their judgments are more difficult to investigate and
will require further study. (Am J Orthod Dentofacial Orthop 2009;135:621-34)
